PROLONGATION OF QUASI-PRINCIPAL FRAME BUNDLES AND 
GEOMETRY OF FLAG STRUCTURES ON MANIFOLDS 

BORIS DOUBROV AND IGOR ZELENKO 

Abstract. Motivated by the geometric theory of difTerential equations and the variational ap- 
proacli to the equivalence problem for geometric structures on manifolds, we consider the problem 
of equivalence for distributions with distinguished submanifolds of flags on each fiber. We call 
them fiag structures. The construction of the canonical frames for these structures can be given 
in the two prolongation steps: the first step, based on our previous works [171 I18| . gives the 
canonical bundle of moving frames for the distinguished submanifolds of flags on each fiber and 
the second step consists of the prolongation of the bundle obtained in the first step. The bun- 
dle obtained in the first step is not as a rule a principal bundle so that the classical Tanaka 
prolongation procedure for filtered structures can not be applied to it. However, under natural 
assumptions on submanifolds of fiags and on the ambient distribution, this bundle satisfies a nice 
weaker property. The main goal of the present paper is to formalize this property, introducing 
the so-called quasi-principle frame bundles, and to generalize the Tanaka prolongation procedure 
to these bundles. Applications to the equivalence problems for systems of differential equations 
of mixed order, bracket generating distributions, sub-Riemannian and more general structures on 
distributions are given. 



1. Introduction 

1.1. Definition of fiag structures. Let A be a bracket generating distribution on a manifold 
QJt, i.e. a vector subbundle of the tangent bundle T9Jt. Assume that for any point 7 € 9Jt a 
submanifold 3^ in a flag variety of the fiber A(7) of A is chosen smoothly with respect to 7. 
We call such structure a flag structure and denote it by (A, {•J'^liean)- We are interested in the 
(local) equivalence problem for these structures with respect to the group of diffeomorphisms of 

The flag structures appear in the natural equivalence problems for differential equations via 
the so-called linearization procedure |10^ ITTl [T5] and in the variational approach to equivalence 
of vector distributions and more general geometric structures on manifolds via the so-called sym- 
plectification/linearization procedure [U [321 US HH [IH]- These procedures are described 
informally in subsections [T2JI3] below and they will be described in the full generality in section 

El 

The cases when the additional structures (for example symplectic or Euclidean structures) 
are given on each space A (7) is also a part of the theory developed below. In the most of the 
applications the dimension of submanifolds 5'^ is equal to one, i.e. each ^'^ is an unparameterized 
curve. The case, when each curve 3^ is parameterized i.e. some parametrization on it is fixed, up 
to a translation, is discussed as well. 

In the most applications the flag structures are not the original objects of study. They arise 
in a natural way from the equivalence problems for other geometric structures, often after some 
preliminary steps. This motivates us to use the gothic letter 9Jt for the ambient manifolds of 
flag structures in order to distinguish them from the smooth manifolds of the original geometric 
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structures. Usually 9Jl is a space of leaves of a foliation on a certain manifold, naturally related to 
the original structure. For example, 9Jt might be the space of solutions of a differential equations or 
the space of extremals of the variational problems naturally associated with a geometric structure. 
This is also the reason why we denote the points of 9J{ by 7, keeping in mind that they represent 
the leaves of a certain foliation. 

1.2. Flag structures in geometric theory of differential equations. The more detailed 
description of this class of problems is given in Examples 1 and 2 of section [3j A differential 
equation is considered in geometry as a submanifold the equation manifold, of the corresponding 
jet space and the solutions (prolonged to this jet spaces) foliate these submanifold. The space of 
solutions Sol can be considered as a quotient manifolds of the equation manifold by this foliation. 
In this case the ambient manifold SOT for the flag structure is taken as the manifold of solutions 
Sol, the distribution A is the whole tangent bundle of SOT, and for a (prolonged) solution 7 the 
curves of flags corresponds to the linearization of the differential equation (or the equation in 
variation) along the solution 7 and geometrically it describes the dynamics of the fibers of £ over 
the jet spaces of lower order by the "flow of solutions" along the solution 7. 

1.3. Flag structures associated with sub-Riemannian and sub-Finslerian structures. 

The more detailed description of this class of problems is given in Example 4 of section [3l By 
a geometric structure on a manifolds M we mean a submanifold of its tangent bundle TM 
transversal to the fibers. 

Let us give several examples. A distribution D on M is given by fixing a vector subspace 
D{q) of TqM, depending smoothly on q. A sub-Riemannian structure U on M with underlying 
distribution D is given by choosing on each space D{q) an ellipsoid U{q) symmetric with respect 
to the origin. In this case U{q) is the unit sphere w.r.t. the unique Euclidean norm on Dq, i.e. 
fixing an ellipsoid in Vg is equivalent to fixing an Euclidean norm on Dg for any q £ M. If in 
the constructions above we replace the ellipsoids by the boundaries of strongly convex bodies in 
D{q) containing the origin in their interior (sometimes also assumed to be symmetric w.r.t. the 
origin) we will get a sub-Finslerian structure on M. In the case when D = TM we obtain in this 
way classical Riemannian and Finslerian structures. 

The key idea, due to A. Agrachev, of the variational approach to the equivalence problem of 
geometric structures (or of the sympelctification of the equivalence problem) is that invariants of 
a geometric structure U on a manifold M can be obtained by studying the flow of extremals of 
variational problems naturally associated with Z// [3ll3]. For this first one can define admissible (or 
horizontal) curves of the structure lA. A Lipschitzian curve a{t) is called admissible (horizontal) 
if a(t) (z U T^(j^^M for almost every t. Then one can associate with lA the following family 
of the so-called time-minimal problems: given two points q^ and qi in M to steer from q^ to qi 
in a minimal time moving along admissible curves of lA. For sub-Riemannian (sub-Finslerian) 
structures these time-minimal problems are exactly the length minimizing problems. 

The Pontryagin Maximum Principal of Optimal Control gives a very efficient way to describe 
extremals of the time-minimal problems. There are two types of Pontryagin extremals of an 
optimal control problem, normal extremals and abnormal extremals. The Lagrangian multiplier 
near the functional of the optimal problem is not equal to zero in the first case and equal to 
zero in the second one. Under certain regularity assumptions that in particular hold for sub- 
Riemannian and sub-Finslerian structures the normal Pontryagin extremals of these variational 
problems foliate certain codimension one submanifold % of the cotangent bundle T*M of the 
ambient manifold M ("H is a non-zero level set of the so-called maximized Hamiltonian of the 
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time-minimal problem). In the case of Riemannian (Finslerian) structures the projections of 
these extremals to the base manifold are classical Riemannian (Finslerian) geodesies. 

How the flag structures appear here? The role of the ambient manifold dJl for the corresponding 
flag structure is played by the space of all normal extremals considered as the quotient of the 
Hamiltonian level set H by the foliation of these extremals, the distribution A again coincides 
with the whole tangent bundle of 91. Besides, the space of all normal extremals ^ is endowed with 
the natural symplectic structure, induced from the canonical symplectic structure on T*M. To 
an extremal 7 a curve Z'^ of Lagrangian subspace of TyCR with respect to this symplectic structure 
can be intrinsically assigned. It corresponds to the linearziation of the flow of extremals along 
the extremal 7, i.e to the Jacobi equation along 7. Therefore it is called the Jacobi curve of 7 
[31 H] . Geometrically it describes the dynamics of the fibers of the Hamiltonian level set "H over 
the base manifold M by the flow of extremals along the extremal 7. Collecting the osculating 
spaces of this curve of any order together with their skew symmetric complements w.r.t to the 
natural symplectic structure one finally assigns to 7 a curve of isotropic/coisotropic subspaces (or 
the curves of symplectic fiags in the terminology of |17] ) in each space A (7) This curve plays the 
role of in the flag structure associated with the geometric structure U and it will be called the 
Jacobi curve of the normal extremal 7 as well. Note also that in the considered case the extremals 
are parameterized by a natural parameter, so the Jacobi curves Z"' are parameterized as well. 

1.4. Flag structures for bracket generating distributions. The more detailed description 
of this class of problems is given in Example 3 of section [31 If as a geometric structure U a 
bracket generating distribution D (without any additional structure on it) is considered, then the 
time-minimal problem associated with U does not make sense: any two points can be connected 
by an admissible curve to D in an arbitrary small time. Instead, one can consider any variational 
problem on a space of admissible curves of this distribution with fixed endpoints. Among all 
Pontryagin extremals in most of the cases there are plenty of abnormal extremals. They do not 
depend on the functional but on the distribution D only. 

Abnormal extremals foliate certain even-dimensional submanifold T-Ld of the projectivized 
cotangent bundle ¥T*M (note that in Example 3 of section 3 we use a different notation for 
"Hd)- The role of the ambient manifold 9Jt for the flag structure is played here by the space 21 
of the abnormal extremals considered as the quotient oiT-Lo by the foliation of these extremals. 
The role of the distribution A is played by the natural contact distribution induced on 2t by the 
tautological 1-form (the Liouville form ) in T*M. The canonical symplectic form is deflned, up to 
a multiplication by a non-zero constant, on each space A (7). The set I-Ld inherits the structure of 
a flber bundle (over M) from Pr*Af . The dynamics of the flbers of Hd by the flow of abnormal 
extremals along the given extremal 7 can be encoded by a curve of isotropic subspaces with re- 
spect to this symplectic form in A (7). Collecting the osculating spaces of this curve of any order 
together with their skew symmetric complements w.r.t to the natural symplectic structure one 
flnally assigns to 7 a curve of isotropic/coisotropic subspaces (or the curves of symplectic flags in 
the terminology of [T7|) in each space A (7) called the Jacobi curve of the abnormal extremal 7. 
This curve plays the role of Z"' in the flag structure associated with the distribution D. 

1.5. Advantages of the passage to flag structures. Why the passage to flag structures via 
the linearization or the sympelctification/linearization procedures is useful and even crucial in 
some cases? 

First of all, making this passage, we immediately arrive to the (extrinsic) geometry of sub- 
manifolds of a flag variety of a vector space W with respect to the action of a subgroup G of 
GL(W), which is simpler in many respects than the original equivalence problem. Assume that 
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we reduced some geometric structure to a flag structure (A, {^'^Iisot) • Then any invariant of a 
submanifold witli respect to tlie natural action of the group GL(A(7)) is obviously an invari- 
ant of the original equivalence problem. Moreover, in general a subgroup of GL(A(7)), which 
is in fact isomorphic to the group of automorphism of the Tanaka symbol of the distribution A 
at 7, acts naturally on A(7) (see subsection 11.71 below for detail). Therefore, any invariant of a 
submanifold of with respect to the natural action of this group, which in general might be 
a proper subgroup of GL(A(7)), is an invariant of the original equivalence problem. In many 
situation this gives a very fast and efficient way to construct and compute important invariants 
of the original structures. 

For example, in the cases of scalar differential equations up to contact transformations (Example 
1 of section [3]) and of rank 2 distributions (a particular case of Example 3 of section [3|) the curves 
^"^ are curves of complete flags that can be recovered by osculation from the curves of their one- 
dimensional subspaces, i.e. from curves in projective spaces. The classical Wilczynski invariants 
of curves in projective spaces [36] immediately produce invariants of the original structures. 

In the case of rank 2 distributions these curves in projective spaces are not arbitrary but they 
are so-called self-dual |15] . In particular, the first nontrivial Wilczynski invariant of self-dual 
curves in projective spaces produces the invariant of rank 2 distributions in which coincides 
with the famous Cartan covariant binary biquadratic form of rank 2 distributions in [71 HUl [S] . 
It gives the new and quite effective way to compute this Cartan invariant and to generalize it to 
rank 2 distributions in R" for arbitrary n > 5 |39| . 

Moreover, the passage from a geometric structure to the corresponding flag structures via the 
linearization or the symplectification/linearization procedures allows one not only to construct 
some invariants but provides an effective way to assign to this geometric structure a canonical 
(co)frame on some (fiber) bundle over the ambient manifold. In some cases this way is much 
more uniform, i.e can be applied simultaneously to a much wider class of structures, than the 
classical approaches such as the Cartan equivalence method (or its algebraic version, developed 
by N. Tanaka [331 El], see also surveys [HUT]) applied to a geometric structure directly. 

Let us clarify the last point. In general, in order to construct canonical frames for geometric 
structures, first, one needs to choose a basic characteristic of these geometric structures, then 
to choose the most simple homogeneous model among all structures with this characteristic, if 
possible, and finally to imitate the construction of the canonical frame for all structures with this 
characteristic by the construction of such frame for this simplest model. And it is desirable that 
the latter can be done without a further branching. The main question is what basic characteristic 
to choose for this goal? 

For example, in the Tanaka theory [33], applied to distributions, as such basic characteris- 
tic one takes the so-called Tanaka symbol of a distribution at a point (see also subsection 11.71 
below). Algebraically a Tanaka symbol is a graded nilpotent Lie algebra. The simplest model 
among all distributions with the given constant Tanaka symbol is the corresponding left-invariant 
distribution on the corresponding Lie group. The construction of the canonical frames for all 
distributions with the given constant Tanaka symbol (i.e. such that their Tanaka symbols at all 
points are isomorphic one to each other as graded nilpotent Lie algebras) can be indeed imitated 
by its construction for the simplest homogeneous model and can be described purely algebraically 
in terms of the so-called universal algebraic prolongation of the Tanaka symbol (see subsection 
12.41 below, especially Theorem 12.21 there) . 

However, it is hopeless in general to classify all possible graded nilpotent Lie algebras and 
the set of all graded nilpotent Lie algebras contains moduli (continuous parameters). Therefore, 
first, generic distributions may have non-isomorphic Tanaka symbols at different points so that 
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in this case Tanaka theory cannot be directly applied and, second, even if we restrict ourselves to 
distributions with a constant Tanaka symbol only, without the classification of these symbols we 
do not have a complete picture about the geometry of distributions. 

Applying the symplectification/linearization procedure to particular classes of distributions (to 
rank 2 distributions in [14j and to rank 3 distributions in |16j ) we realized that we can distinguish 
another basic characteristic of a distribution, which is coarser than the Tanaka symbol and, more 
importantly, classifiable and does not depend on continuous parameters. This is the so-called 
symbol of the Jacobi curve of a generic abnormal extremal of the distribution at the generic point 
or shortly, the Jacobi symbol of a distribution (see our recent preprint [21j for more detail). The 
notion of the symbol of a submanifold in a flag variety at a point was introduced in [n\ I18| . 
see also subsection 12.31 below. Algebraically such symbol is a subspace (a line in the case of 
curves) of degree —1 endomorphisms of a graded vector space, up to a natural conjugation by 
endomorphisms of nonnegative degrees, i.e. it is much more simple algebraic object than a Tanaka 
symbol. Informally speaking, it represents a type of the tangent space to the submanifold of flags. 

For Jacobi curves of abnormal extremals the symbol at a point is a line of a degree —1 of 
a so-called graded symplectic space (see [T71 subsection 7.2]) from a symplectic algebra of this 
space. All such lines, up to the conjugation by symplectic transformations, were classified in [T71 
subsection 7.2]. In particular, the set of all equivalence classes of such lines is discrete. This 
in turn gives the classification of all Jacobi symbols of distributions and leads to the following 
new formulation: to construct uniformly canonical frames for all distributions with given constant 
Jacobi symbols. 

This problem leads in turn to a more general problem of construction of canonical frames for flag 
structures with given constant flag symbol and it was the main motivation for all developments 
of the present paper. The solution of the latter general problem is given by Theorem 12.41 below, 
which is the direct consequence of the main result of the present paper, given by Theorem 12.31 
Theorem 12.41 shows that the construction of canonical frames for distributions can be described in 
terms of natural algebraic operations on the Jacobi symbols in the category of graded Lie algebras. 
Note that from the fact that the set of all Jacobi symbols of distributions is discrete it follows 
that the assumption of the constancy of a Jacobi symbol holds automatically in a neighborhood 
of a generic point of an ambient manifold. More detailed applications of Theorem 12.41 to the flag 
structures appearing after the symplectification/linearization procedure of distributions will be 
given in j21j . 

1.6. Flag structures and G-structures on filtered manifolds. Classical G-structures are 
defined as the reduction of the principal frame bundle J^{W) to a certain subgroup G in GL{V), 
where ^ is a model tangent space to 9Jt. In many cases the action of G on a variety of flags of V 
has a unique closed orbit 5, and G itself can be recovered as a symmetry group of this orbit. 

A typical example is the irreducible action of GL{2,M) on any finite-dimensional vector space 
V. The induced action of GL(2,R) on the projectivization P{V) has a rational normal curve as 
a unique closed orbit. 

Assume that A coincides with the tangent bundle TdJl. We say that the flag structure is of type 
5, if its fibers are equivalent to 5 at all points of M. As the orbit 5 in the flag variety defines the 
subgroup G C GL{V) uniquely an vice versa, we see that there is a one-to-one correspondence 
between flag structures of type 5 and G-structures on M. For example, GL(2, ii)-structures 
are defined as reductions of the principal frame bundle to the irreducible subgroup GL{2,R) of 
GL{y). Equivalently, in our terminology they are flag structures defined by a family of rational 
curves Z'^ C P(r^9Jt) smoothly depending on the point 7. For more details on GL(2, i?)-structures 
and their relationship with invariants of ODEs see |1H [22t [23} [2^ . 
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Flag structures of type 5 constitute a very special type of flag structures we study here, be- 
cause we consider quite arbitrary distributions A (with constant Tanaka symbol) and we do not 
assume that the submanifolds ^'^ are isomorphic at different points. However, in some sense, we 
imitate the construction of canonical frame for general flag structures by approximating them by 
the flag structure with submanifolds ^'^ being the orbits of a certain subgroup of the group of 
automorphism of the Tanaka symbol of A. This allows us to combine together both our version 
of construction of moving frames for submanifolds in flag varieties |18l [T7] and the prolongation 
theory for G-structures on filtered manifolds [51 133[ 1^. As a result we get a powerful technique 
for solving the local equivalence problem for arbitrary flag structures. 

1.7. Construction of canonical frames for flag structures: preliminary steps and dis- 
cussions. After considering separately the equivalence problems for several particular classes 
of differential equations and geometric structures via the linearization and the symplectifica- 
tion/linerization procedure we arrived to the necessity to develop a general approach to the 
equivalence problem of flag structures. 

Let (A, {■H'^Iisot) be a flag structure. To begin with, let us discuss the geometry of the 
distribution A itself. Let A^^ C A~^ C ... be the weak derived flag (of A), defined as follows: Let 
Xi, . . . Xihel vector fields constituting a local basis of a distribution A, i.e. A = spanjXi, . . . ,Xi} 
in some open set in 9Jt. Then A".' (7) is the linear span of all iterated Lie brackets of these vector 
fields, of length not greater than j, evaluated at a point 7. 

The basic characteristic of a distribution A at a point 7 is its Tanaka symbol. To define it let 

0~^(7) =^^"^(7) ^ud g''(7) =^ A.^(7)/A.^"'"^(7) for j < —1. Consider the graded space 

-1 

(1-1) m(7) = 0^(7), 

j=-^J■ 

corresponding to the filtration 

A(7) = A-i(7) c A^2(7) c . . . C A-'^+i(7) c A-'^(7) = T^M. 

This space is endowed naturally with the structure of a graded nilpotent Lie algebra, generated 
by 0^^(7). Indeed, let pj : A.'(7) i-)- 0.'(7) be the canonical projection to a factor space. Take 
^1 £ 0^(7) ^ud Y2 S 0-'(7). To define the Lie bracket [11,12] take a local section Yi of the 
distribution A* and a local section I2 of the distribution A-' such that pj(li(7)) = li and 
Pj {%{!)) = I2. It is clear that [Yi,Y2] G g^+^ (7). Put 

(1.2) [yi,y2] =Pi+,([?i,?2](7)). 

It is easy to see that the right-hand side of ()1.2p does not depend on the choice of sections Yi 
and I2. Besides, 0~^(7) generates the whole algebra tn(7). A graded Lie algebra satisfying the 
last property is called fundamental. The graded nilpotent Lie algebra m(7) is called the Tanaka 
symbol of the distribution A at the point 7. 

For simplicity assume that the Tanaka symbols m(7) of the distribution A at the point 7 are 

-1 

isomorphic, as graded Lie algebra, to a fixed fundamental graded Lie algebra m = g*. In this 

i=—fi 

case A is said to be of constant symbol m or of constant type m. Note that in all our motivating 
Examples 1-4 given in section [3] the distribution A is of constant type: in Examples 1, 2, and 4 
A = T9Jt and the symbol m is graded trivially, m = g~^, i.e. m is the commutative Lie algebra 
of dimension equal to dimOJt; in example 3 A is a contact distribution on lUt and its symbol is 
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isomorphic to the Heisenberg algebra of dimension equal to dimSIJt with the grading © , 
where is the center. 

Note also that one can distinguish the so-called standard or flat distribution of constant 
type m. For this let M(m) be the simply connected Lie group with the Lie algebra m and let e 
be its identity. Then Dm is the left invariant distribution on M(m) such that -Dtn(e) = Q~^- The 
distribution Df^ is in a sense the most simple one among all distributions of constant type m. 
Note that in all our motivating Examples 1-4 given in section [3] the distribution A has a trivial 
local geometry, i.e. it is locally equivalent to the flat distribution with the same symbol. However, 
we do not need to assume such local triviality to develop our theory. 

Further, to a distribution A with constant symbol m one can assign a natural principle bundle 
over 9JT. Let G^{m) be the group of automorphisms of the graded Lie algebra m; that is, the 
group of all automorphisms A of the linear space m preserving both the Lie brackets (^([f , w]) = 
[A{v),A{w)] for any v,w e xn) and the grading (A(g*) = for any i < 0). Let P°(m) be the 
set of all pairs (7,92), where 7 E 9Jt and 99 : m — )• 1x1(7) is an isomorphism of the graded Lie 
algebras m and m(7). Then P^{m) is a principal G*^ (m)-bundle over 9Jt. The right action Ra 
of an automorphism A G 0*^(1x1) is as follows: Ra sends (7, (/j) S P^{m) to {'y,^p o A), or shortly 
(7,99) • Ra = (7, 95 o A). Note that since g~^ generates m, the group G''(m) can be identified 
with a subgroup of GL(g~^). By the same reason a point (7,99) € P^{m) of a fiber of P°(m) is 
uniquely defined by 9?|g-i- So one can identify P''(m) with the set of pairs {-Jjip), where 7 € 9Jt 
and ip : g~^ ^il) can be extended to an automorphism of the graded Lie algebras m and m(7). 
Speaking informally, P'^(m) can be seen as a G''(m)— reduction of the bundle of all frames of the 
distribution A. Besides, the corresponding Lie algebra g''(m) is the algebra of all derivations a of 
m, preserving the grading (i.e. ag* C g* for all i < 0) and it can be identified with a subalgebra 
ofgl(g-i). 

If A = Tm (as in Examples 1, 2, and 4 of section E]), then G°(m) = GL(m), and P°(m) 
coincides with the bundle J-'(5Jt) of all frames on DJl. In this case is nothing but a usual 
G'^(m)-structure. If A is a contact distribution (as in example 3), then a non-degenerate skew- 
symmetric form Q, is well defined on g~^, up to a multiplication by a nonzero constant. The 
group G^{m) of automorphisms of m is isomorphic to the group CSP(g~^) of conformal symplectic 
transformations of g~^, i.e. transformations preserving the form U, up to a multiplication by a 
nonzero constant. 

Note that on each fiber A (7) a group of automorphism of the symbol 1x1(7) acts naturally. 
Obviously, by constructions G^ is a subgroup of GL[D{'^)^ and it is isomorphic by conjugation 
to 0*^(1x1). For example, in the case when A is contact GlJ is a group of all conformal symplectic 
transformations of A(7) with respect to the natural conformal symplectic structure on A(7). 

Additional structures on the distribution A can be encoded as fiber subbundles of the bundle 
P''(m). Since in our case the submanifolds of fiags on each fiber of A are given, it is natural to 
fix a filtration on the space g~^ (nonincreasing by inclusion): 

(1-3) {9j^}j&, Qj^ C Qj\, Qj^ C g-i, dim qJ^ = dim ^]{x),x € 7. 

Then we can consider the subbundle Pj(m) of P^{m) consisting of the pairs (7,99) such that 
99 : m — > tn(7) is an isomorphism of the graded Lie algebras m and m(7) such that the flag 
{(p[gj^)}j(zz belongs to the submanifold of flags ^'^ . 

Next step is to assign to the flag structure (A, {d'^}y£<sx) ^ subbundle of P^(m) of the minimal 
possible dimension. For this one needs to take a closer look to the (extrinsic) geometry of sub- 
manifold of flags ^'^ with respect to the natural action of the group G^ (~ G'^(m)). In [171 IIH] 
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we developed an algebraic version of Cartan's method of equivalence or an analog of Tanaka pro- 
longation for submanifold of flags of a vector space W with respect to the action of a subgroup 
G of GL(W). Under some natural assumptions on the subgroup G and on the flags (that will 
be discussed below in detail and that are valid in all our motivating examples), one can pass 
from the filtered objects to the corresponding graded objects and describe the construction of 
canonical bundles of moving frames for these submanifolds in the language of pure linear algebra. 
Applying this construction to each submanifold F^, one gets the desired distinguished subbundle 
P of pO(m). 

The important point here is that this subbundle P is as a rule not a principal subbundle of the 
bundle P^{m). For example, if the subgroup of G^ preserving the submanifold ^'^ , i.e. the group 
of symmetries of 2'^ from G^, does not act transitively on 5'^, then P is not a principal subbundle 

of the bundle P^ (m) . Note that from the constructions of [18^ [T7] it follows that if 971 denotes 
the so-called tautological bundle over 9Jt, i.e the bundle with the fibers over the 7 € 9Jl consisting 
of the points of the curve 3'^, then P can be considered also as the bundle over 9K. It turns out 
that P can not always be chosen as a principal bundle over 9K as well. So, the classical Tanaka 
prolongation procedure for the construction of the canonical frame for principal reductions of the 
bundle P^{m) ( [6l [331 fl^ ) in general can not be applied here. However, due to the presence of the 
additional filtration ()1.3p on the space the subbundle -P — )• 9Jl satisfies a nice weaker property 
of constancy of the graded spaces corresponding to the tangent spaces to the fibers of P. 

The main goal of the present paper is to formalize this property, introducing the so-called 
quasi-principle subbundles of P^{m) (Definition 12. 2p . and to generalize the Tanaka prolongation 
procedure to the quasi-principle subbundles (Theorem I2.3p of P'^(m). As a consequence we ob- 
tain a general procedure for the construction of canonical frames for flag structures (Theorem 
12. 4p . In section [3] we give applications of this theory in a unified way (Theorem 13. ip to natural 
equivalence problems for various classes of differential equations, bracket generating distributions, 
sub-Riemannian and more general structures on distributions. Sections H] and [5] are devoted to 
the proof of the main Theorem 12.31 

2. Quasi-principle frame bundles and main results 

2.1. Compatibility of flags with respect to the grading. First, let us recall some basic 
notions on filtered and graded vector spaces. Here we follow [n\ section 2]. Let {Aj}j^z be 
a decreasing (by inclusion) filtration (flag) of a vector space W: Aj C Aj„i. It induces the 
decreasing filtration {{Ql{W))i}i^z of 0l(VF), 

(2.1) {QKW))i = {Ae Ql{W) : A{Aj) C A,+i for ah j}, {Ql{W))i C {qI{W))^^i. 

It also induces the filtration on any subspace of g[(VF). Further, let grW be the graded space 
corresponding to the filtration {Ajjjgx, 

grI^ = 0A,/Ai+i 

and let gr gl(Vl^) be the graded space corresponding to the filtration ()2.ip . 

grflW = 0(0l(VF)),/(0l(VF)),+i. 

The space gr 0l(l^) can be naturally identified with the space Qi{gicW), 

(2.2) gv Ql{W)^gi{grW). 
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Indeed, if Ai and A2 from {gl{W))i belong to the same coset of {Ql{W))i/ {Ql{W))i+i, i.e. A2—A1 £ 
{gl{W))i+i, and if wi and W2 from Aj belong to the same coset of Aj/Aj+i, i.e. W2 — wi G ^j+i, 
then Aiwi and A2W2 belong to the same coset of Aj+j/Aj+j+i. This defines a linear map from 
gr g[(VF) to Q\{gxW). It is easy to see that this linear map is an isomorphism. 

Now let g be a Lie subalgebra of 01(1^). The filtration induces the filtration {gi}i<^z 

on 5, where 

Let gr g be the graded space corresponding to this filtration. Note that the space gi/giJ^i is 
naturally embedded into the space (g[(VF))j/(g[(Vl^))j+i. Therefore, gr g is naturally embedded 
into gr gl(M^) and, by above, gr g can be considered as a subspace of 0[(gr 14^). It is easy to see 
that it is a subalgebra of g[(gr M/^). 

In general, the algebra gi g is not isomorphic to the algebra g (see Example 2.1]). In order 
to develop an algebraic version of the Cartan prolongation procedure it is very important that the 
passage to the graded objects will not change the group in the equivalence problem. Therefore 
we have to impose that gr g and g are conjugate as in the following 

Definition 2.1. We say that the filtration (the flag) {Ajjjg^ ofW is compatible with the algebra 
g C 0[(VF) with respect to the grading if there exists an isomorphism J : giW W such that 

(1) Ji^i/A^+l) C Ai, i e Z; 

(2) J conjugates the Lie algebras gr g and g i.e. 

(2.3) g = {JoXoJ~^:XegTg}. 

Remark 2.1. Obviously, if G = GL(W) or SL(W), then any filtration of W is compatible with 
gl(II^) or s[(VF) with respect to the grading. On the other hand, if W is endowed with a symplectic 
form or a conformal symplectic form (i.e. a symplectic form defined up to a multiplication by 
a nonzero constant) and G = Sp{W) or CSp{W), then, according to [TTJ Proposition 2.2], a 
decreasing filtration {Aj}j^z of W is compatible with sp(T4^) or csp(iy) if and only if, up to a 
shift in the indices, (gZj)^ = g~3,y for some integer u, where L'^ denotes the skew-symmetric 
complement of a subspace L with respect the symplectic form on W {u can be taken as or 1). 
Filtration (fiags), satisfying this property are called symplectic filtrations (flags). Note that all 
flags appearing in Examples 3 and 4 are symplectic. □ 

2.2. Quasi-principal subbundles of i-''^(m). Now we are ready to introduce the notion of 

a quasi-principle bundle. Let, as before, A be a distribution with a constant Tanaka symbol 
-1 

m = g* and assume that the space g~^ has an additional filtration 

i=—li 

(2.4) {Qf}jez, Qj' C qJ\, qJ^ Q 

Let the bundle -P'^(tn) be the bundle defined in subsection 11.71 Let P be a fiber subbundle of 
P°(m) and ^(7) be the fiber of P over the point 7. Take ip G P{'y). The tangent space T^[P{'y)) 
to the fiber P{'y) at a point ip can be identified with a subspace of 0*^(111). Indeed, define the 
following 0'^(m)-valued 1-form O on P: to any vector X belonging to r^(P(7)) we assign an 
element Q{ip){X) of 0°(m) as follows: if s — > ip{s) is a smooth curve in ^(7) such that ip{0) = ip 
and 99' (0) = X then let 

(2.5) n{ip){X) = ip-^ o X, 
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where in the right hand side of the last formula (p is considered as an isomorphism between m 
and m(7) . Note that the linear map VL{lp) : r^(P(7)) fl°(m) is injective. Set 

(2.6) Ll:=n{T^{P{t)) 

If P is a principle bundle, which is a reduction of the bundle P^(m) and C 0'^(m) is the 
Lie algebra of the structure group of the bundle P, then the space is independent of ip and 
equal to g". We call this bundle P a principle bundle of type (m, g'') or a Tanaka structure of type 
(m, Q^) (as in [6]). For general subbundle P subspaces may vary from point to point. From the 
constructions of the previous subsection, the filtration ()2.4p induces the filtration on each space 
defined in ()2.6p and the corresponding graded spaces gr can be considered as subspaces of 
0[(grg~^). In many important applications these subspaces are independent of ip. This motivates 
the following 

Definition 2.2. Assume that is a subalgebra of g^{m). A fiber subbundle P of the bundle 
P'^(m) is called a quasi-principle bundle of type (m, g'^) if the following three conditions hold: 

(1) The filtration (|1.3p o/g"^ is compatible with the algebra Q^{xn) with respect to the grading; 

(2) The subspaces gr o/g[(grg~^) coincide for any p € P; 

(3) There exist an isomorphism J : grg~^ i— >■ as in Definition \2.1\ (with W = Q~^) such 
that g° = { J o X o J-i : X G gr L° } for any p € P. 

Note that if the filtration (jl.Sp on is trivial (i.e. it does not contain any nonzero proper 
subspace of g~^) then P is, at least locally, the principle bundle of type (m, g*^) in the Tanaka 
sense. 

2.3. Quasi-principal bundles associated with flag structures with constant flag symbol. 

Our next goal is to show that under some natural assumptions and based on |17j . one can assign 
to a structure (A, {2'^}"f^on) a quasi-principal bundle in a canonical way. 

Recall that a group (conjugate to G°(m)) acts naturally on each A(7). Let be the Lie 
algebra of the Lie group G°. Assume that the submanifolds ^'^ satisfy the following additional 
properties 

(Fl) (transitivity) All flags in lie in the same orbit under the natural action of the group 

(F2) ( compatibility with respect to the grading) Any flag in 5'^ is compatible with with respect 
to the grading. 

Then, flrst we can fix a filtration on the space as in (|1.3p such that it is compatible with 
g''(m) with respect to the grading. For this it is enough to fix one of the flags ^^{x) in the space 
A(7) and to identify somehow the space g~"^ and the space A(7) with this fixed flag. 

Further we will assume that the submanifolds of flags are not arbitrary but satisfy a special 
property called the compatibility with respect to differentiation in the terminology of \n\ Section 
3]. 

First describe this property for curves of flags. Given a curve t ^ L{t) in a certain Grassman- 
nian of a vector space W , denote by C{L) the canonical vector bundle over L: the flber of C{L) 
over the point L[t) is the vector space L{t). Let T[L) be the space of all sections of the bundle 
C(L) . Set L(°)(t) := L{t) and deflne inductively 

L^^\t) = span{^i{t) : i G r(L),0 < k < j} 
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for j > 0. The space L^^\t) is called the jth extension or the jth osculating subspace of the curve 
L at the point t. With this terminology, relation (|3.ip of property (P3) implies the following 
property 

(F3) If 5'*' is a curve of flags, we assume that for any i € Z the first extension (the first 
osculating subspace) of the curve ^]{x),x € 7 at any point xq G 7 is contained in the space 

(2.7) (37)^'H^o) ca7_i(xo), yxoej,ieZ. 

This relation exactly means that the curve 5'*' = {5j(x)}j(=z is compatible with respect to differ- 
entiation in the sense of [T7]. If is a submanifold of flags of arbitrary dimension, we assume 
that any smooth curve on it is compatible with respect to differentiation as above. 

Now let G^im.) be the subgroup of G'^(m) consisting of all elements of G'^(m) preserving the 
filtration (|2.4p . The group G5j_(m) acts naturally on the space (gr£|'^(m)) of all degree —1 endo- 
morphisms of the graded space grg"^ (corresponding to the filtration ()1.3p ) belonging to grg''(m). 
This action is in assence the adjoint action, namely A £ G^{m) sends x S (gr0''(m)) to the 
degree —1 component of (Ad A) x. This action induces the natural action on the Grassmannians 
of (gr0°(m))_i. 

Further recall that given a Grassmannian of subspaces in a vector space W the tangent space 
at any point (= a subspace) A to this Grassmannian can be naturally identified with the space 
Hom(A, VF/A). Taking into account that the submanifolds of flags ^'^ satisfies the compatibility 
with respect to differentiation property (F3), we can conclude from here that the tangent space 
to the submanifold at a point ^'^{x) can be identified with a subspace in the space 

Hom(a7 (x)AZ+i (x) , Ci (^) (^)) 

or, in other words, a subspace in the space of degree —1 endomorphisms of the graded space 
corresponding to the filtration ^"^{x). 

Moreover, taking into account properties (Fl) and (F2) and the construction of the filtration 
(|1.3p . we can identify the tangent space to the submanifold at a point 2'^{x) with a subspace 
62 of the space (grg'^(m)) of all degree —1 endomorphisms of gTQ~^ from gr0''(m), defined up 
to the aforementioned natural action of the group G^{m). This subspace (or, more precisely, the 
orbit of this subspace with respect to this action) is called the symbol of the submanifold ^-y at 
the point X'{x). 

The symbols 6j play in the geometry of submanifolds of flags the same role as Tanaka symbols 
in the geometry of distributions. Note that the symbol 6x must be a commutative subalgebra 
of gr0^(m) belonging to (grg'^(m)) The latter condition follows from the involutivity of the 
tangent bundle to the submanifold 5'^. 

Finally, we need the following property 

{F 4) (constancy of the flag symbol) Symbols 52 of the submanifold of flags at a point are 
independent of x and 7 or, more precisely, lie in the same orbit under the action of the group 
G^{m) on the corresponding Grassmannian of the space (gr3'^(m)) If (5 is a point in this orbit 
we will say that the structure (A, {^'^l^ggrji) has the constant flag symbol 6 (and the constant 
Tanaka symbol m). 
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Remark 2.2. Under some natural assumptions the condition of constancy of the flag symbol 
is not restrictive at least in the case when ^7 are curves of flags. If G'^(m) is semismple (or, 
more generally, reductive), as direct consequence of results E.B. Vinberg ([35]) the set of all 
possible symbols of curves of flags is finite. Hence, the symbol of a curve of flags with respect to a 
semisimple (reductive) group G is constant in a neighborhood of a generic point. Note also that all 
flag symbols that may appear in Examples 1-4 of section [3] are classified in [T7], see subsection 7.1 
there, corresponding to Examples 1 and 2 , and subsection 7.2 there, corresponding to Examples 
3 and 4. □ 

As in Tanaka theory, one can define the notion of the flat submanifold of flags with constant 
symbol 5. Taking into account the identification between the spaces grg~^ and g"^, consider the 
subgroup H{5) of G^{m) with the Lie algebra 5. The flat submanifold with constant symbol 5 is a 
submanifold equivalent to the orbit of the flag (|2.4|) with respect to the natural action of Hg on 
the corresponding flag variety. 

Again by analogy with the Tanaka theory, one can define the universal algebraic prolongation 
of the flag symbol 6 (for the description of the algebraic prolongation of a Tanaka symbol see 
the next subsection and also [331 EH SI])- Let (grg*'(m))^, be the component of degree k of the 
space gr0*^(m) or equivalently the endomorphisms of degree k of grg~^ belonging to gr0''(m). Set 
'■= S and define by induction in k 

(2.8) uf (5) := {X G (gr0°(m)), : [X,Y] G uti{S), Y e 6}, k > 0, 

where (gr0'^(m))^ denotes the space of all degree k endomorphisms of the graded space grQ~^ 
(corresponding to the filtration p.3|) ) belonging to grg'^(m). The space u^{S) is called the kth 
algebraic prolongation of the symbol 6. Then by construction 

(2.9) u^{6)= 0uf(5) 

fc>-i 

is a graded subalgebra of gr0'^(m). It can be shown that it is the largest graded subalgebra of 
the space ffQ^{m) such that its component corresponding to the negative degrees coincides with 
5. The algebra u^{5) is called the universal algebraic prolongation of the flag symbol 5 (of a 
commutative subalgebra of grg''(m) belonging to (grg''(m)) _-|^). As it is shown in [15], the algebra 
of infinitesimal symmetries of the flat submanifold with the constant symbol 5 (with respect to 
the action of the group G'^(m)) is isomorphic to [5). 

Since by the constructions the filtration ()1.3p is compatible with 0^(m) with respect to the 
grading, the subalgebra u^((5) of grg'^(m) is conjugate to a subalgebra of 0''(tn) (see condition (2) 
of Definition 12. ip . Taking into account this conjugation, from now on we will look on u^(5) as 
on the subalgebra of 0'^(m). The procedure for the construction of a canonical bundle of moving 
frames for a submanifold in a flag variety with a constant symbol is described in \n\ I18| . As a 
direct consequence of |17^ Theorem 4.4] in the case of curves of flags and its generalization to 
submanifolds of flags described in subsection 4.6 there, applied to each submanifold 3'^, we obtain 
the following 

Theorem 2.1. Given a structure (A, {J'^j^ggji) with the constant Tanaka symbol m (of A) and 
with the constant flag symbol 6 one can assign to it in a canonical way a quasi-principal subbundle 
o/pO(m) of type {m,u^{6)). 

Note that the assignment in Theorem 12.11 is uniquely determined by a choice of a so-called 
normalization condition for the structure equation of the moving frames associated with the 
submanifolds 5'^. In the case of curves the normalization conditions are given by a subspace W 
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complementary to the subspace u^{6) + [5, g']_(m)] in 0'].(m) , where S+(tn) is the Lie algebra of 
the group G'^(m) and u^{6) = u{6) D g5|_(m). Similar description of normalization conditions can 
be given in the case of submanifolds of flags [iTl [18] . 

The important point is that as a rule the resulting quasi-principal subbundle P of P''(m) 
in Theorem 12.11 is not a principle bundle of type (m,u^((5)). For example, if the subgroup of 
preserving the submanifold 5"^, i.e. the group of symmetries of from G^, does not act 
transitively on 5"^, then P is not a principle bundle of type (m, u^(5)). 

Remark 2.3. (On nice flag symbols.) Let P denote the resulting quasi-principal subbundle of 
Theorem 12.11 One can distinguish a special class of flag symbols for which the normalization 
conditions for the construction of the bundle P can be chosen such that P has a nicer property 
at least on the level of the submanifolds of flags. More precisely, assign to our flag structure 
(A, {T}'re<m) the bundle Tl over dJl such that the fiber over 7 E 5Jt consists of the points of 
the curve . Let us call this bundle the tautological bundle over Then from the same \17\ 
Theorem 4.4] it follows that P can be considered as the bundle over OJl as well. 

However, even the bundle P — >• OJl can not always be chosen as a principle bundle. To formulate 
the precise conditions for the latter, let U^{S) be the subgroup of the group of symmetries of the 
flat submanifold with the constant symbol 6 (with respect to the action of the group G*^(m)), 
which preserves the filtration ()2.4p . The bundle P — > 9Jt is the principal bundle if and if the 
complementary subspace W defining the normalization condition for the construction of P is 
invariant with respect to the adjoint action of U^{6). In the latter case Uf{d) is the structure 
group of this bundle. If such complementary space W can be chosen then the symbol 6 and the 
normalization condition W will be called nice. The fact that not any symbol of submanifolds 
of flags is nice is the main reason why we need to introduce the quasi-principle bundles and to 
develop the prolongation procedure for them. 

For example, the symbol of curve of flags obtained after the linearization procedure for scalar 
ordinary equations (Example 1 of section 3) and symplectification procedure for rank 2 distribu- 
tions (a particular case of Example 3 of section 3) are nice. In both cases the curves of flags are 
generated by osculation from certain curves in projective spaces. However, this is not in general 
the case for systems of ODEs of mixed order and distributions of rank greater than 2. See [IB] 
for the proof of non-existence of nice (in the above sense) normalization conditions for curves of 
flags appearing in ODEs of mixed order (3, 2) up to a 0-equivalence (in the sense of Example 2 of 
section 3 below). Similarly, it can be shown that a nice normalization conditions does not exist 
in the case of rank 3 distributions in M^, having a 6 dimensional square. The questions on the 
classification of flag symbols, for which a nice normalization condition can be chosen, remains 
open. □ 

2.4. Prolongation of quasi-principal frame bundles. Now we return to general quasi-principle 
bundles. In [33] for any principle bundle of type (m, g") Tanaka described the prolongation pro- 
cedure for the construction of the canonical frame (the structure of an absolute parallelism) by 
means of the so-called universal algebraic prolongation of the pair (m, g*^). One of the main goals 
of the present paper is to generalize this prolongation procedure to quasi-principle bundles of type 
(m,0O). 

Now let us define the algebraic prolongation of the pair (m, g*^) and formulate our main result. 
First note that the subspace m © g*^ is endowed with the natural structure of a graded Lie algebra. 
For this we only need to define brackets [/, v] for f £ and f G m, because m and g° are already 
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Lie algebras. Set 

(2.10) [f,v]:=f(v). 

Since is a subalgebra of the algebra of the derivations of m preserving the grading, such 
operation indeed defines the structure of the graded Lie algebra on m © g''. Informally speaking, 
the Tanaka universal algebraic prolongation of the pair (m, g*^) is the maximal (nondegenerate) 

graded Lie algebra, containing the graded Lie algebra g* as its non-positive part. More 

i<0 

precisely, Tanaka introduces the following 

Definition 2.3. The graded Lie algebra u(m, g*') = ^^u*(m, g'^), satisfying the following three 

conditions: 

(1) u*(m,g°) = for all i < 0; 

(2) ifX G u*(m,g°) with i > satisfies [X,g~i] = 0, then X = 0; 

(3) u(m, Q^) is the maximal graded Lie algebra, satisfying Properties 1 and 2. 

is called the universal algebraic prolongation of the graded Lie algebra mffig*^ or of the pair (m, g^). 

The algebra u(m,g'^) can be explicitly realized as follows. Define the k-th algebraic prolongation 
of the Lie algebra m(BQ^ by induction for any k > 0. Assume that spaces g' C Hom(g*, g*^^) 

are defined for all < / < fc. Set 



(2.11) [f,v] = -[v,f] = fiv) yf eg^O <l<k, and V em. 
Then let 

(2.12) g'^ "^'Ife 0Hom(0\g*+'=) : f{[vi,V2]) = [f{vi),V2] + [vi,f{v2)] V^i,t;2 G ml . 

I i<0 J 

Directly from this definition and the fact that m is fundamental (that is, it is generated by g~^) 
it follows that if / G g^ satisfies /|g-i = 0, then / = 0. The space ©jg^g' can be naturally 
endowed with the structure of a graded Lie algebra. The brackets of two elements from m are as 
in m. The brackets of an element with non-negative weight and an element from m are already 
defined by (j2.1ip . It only remains to define the brackets [/i, /2] for /i G g'^, /2 G g' with k,l > 0. 
The definition is inductive with respect to k and /: if A: = / = then the bracket [/i, /2] is as in 
g''. Assume that [/i, /2] is defined for all fi G Q^, /2 G g^ such that a pair {k, I) belongs to the set 

{{k, l):0<k<k,0<l< l}\{{k, I)}. 
Then define [/i, /2] for fi G g^, /2 G g'" to be the element of ^ Hom(g*, g*+''+^) given by 

i<0 

(2.13) [fi,f2]v = [fi{v),f2] + [fij2{v)] yvem. 

It is easy to see that [/i,/2] G g'^^' and that 0jg^g* with bracket product defined as above is a 
graded Lie algebra. As a matter of fact ([33] §5) this graded Lie algebra satisfies Properties 1-3 
of Definition 12.31 . That is it is a realization of the universal algebraic prolongation g(m, g'') of the 
algebra m © g°. In particular u*(m, g°) = g*. 

It turns out ( j33j §6, |37j §2) that u(m,g'') is actually isomorphic to the Lie algebra of infinites- 
imal symmetries of the so-called fiat principle bundle of type (m,g^) . To define this object let, as 
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before, be the flat distribution of the constant type m, i.e. the left invariant distribution on 
M(m) such that -Dm(e) = Q~^- Let be the simply-connect Lie group with the Lie algebra G^. 
Denote by Lx the left translation on M(m) by an element x. Finally, let P'^(m, g*^) be the set of 
all pairs {x, ip), where x S M(m) and ip : m —?■ m{x) is an isomorphism of the graded Lie algebras 
m and m(x) such that {Lx-i)*^p € G^. The bundle P^(m,Q^) is called the flat principal bundle of 
constant type (m, g'^). If dimu(m,g'^) is finite (which is equivalent to the existence of Z > such 
that u'(m,g'') = 0), then the algebra of infinitesimal symmetries is isomorphic to u(m, g''). The 
analogous formulation in the case when u(m, g'') is infinite dimensional may be found in [33] §6. 
One of the main results of Tanaka theory [33] can be formulated as follows: 

Theorem 2.2. Assume that the universal algebraic prolongation u{xn,Q^) of the pair (m.,Q^) is 
finite dimensional and I > is the maximal integer such that u'(m, g'') ^ 0. To any principle 
bundle of type (m, g'') one can assign, in a canonical way, a sequence of bundles {P^}[^q such 
that is an afflne bundle over p^~^ with fibers being affine spaces over the linear space u*(m, g'^) 
(= g') for any i = I, . . .1 and such that P^ is endowed with the canonical frame. 

Once a canonical frame is constructed, the equivalence problem for the principle bundles of 
type (m, g'^) is in essence solved. Moreover, dimu(m, g*^) gives the sharp upper bound for the 
dimension of the algebra of infinitesimal symmetries of such structures. 

The main results of the present paper is the following 

Theorem 2.3. Assume that the universal algebraic prolongation u(m, g*^) of the pair (m, g'^) is 
finite dimensional and I > is the maximal integer such that u^(m, g'^) ^ 0. To any quasi-principle 
bundle P^ of type (m, g'^) one can assign, in a canonical way, a sequence of bundles {P*}'^q such 
that is an afflne bundle over p^~^ with flbers being afflne spaces over the linear space of 
dimension equal to dim u*(m, g°) (= dimg*) for any i = 1,. . .1 and such that P^ is endowed with 
the canonical frame. 

Note that in both theorems above the construction of the sequence of bundles depends on 
the choice of the so-called normalization conditions on each step of the prolongation procedure, 
while in Theorem 12.31 it also depends on the choice of the so-called identifying spaces on each step 
of the prolongation procedure. The precise meaning of these notions will be given in the course of 
the proof of Theorem 1 2. 3 1 in sections H] and [5l This proof is a modification of the proof of Theorem 
12.21 given in the second author's lecture notes |41j . 

As a direct consequence of Theorems 12.11 and 12.31 we get the following 

Theorem 2.4. Given a flag structure (A, {^'^l^g^) with the constant Tanaka symbol m (of A) 
and with the constant flag symbol 5 such that the universal algebraic prolongation u(m,u^(5)) of 
the pair (m, u^(5)) is finite dimensional, one can assign to it a bundle over dJl of the dimension 
dim u(m, U'^(5)) endowed with the canonical frame. 

Thus, the construction of the canonical frames for a flag structure (A, {^^y^^^) with the 
constant Tanaka symbol m (of A) and with the constant flag symbol 5 is reduced first to the 
calculation of the algebra u^(5) and then to the calculation of the algebra u(m,u^((5)) . Both task 
are the problems of linear algebra. 

The last theorem can be applied directly to the structures of subsections ll.2l and ll.4l as discussed 
in more detail in Examples 1-3 of the next section. To apply our scheme to the structures discussed 
in subsection 11.31 we need to make two straightforward modifications, described in two remarks 
at the end of this section. 

In any case for all structures mentioned in subsections II. 2m. 41 ( and discussed in more detail 
in Examples 1-4 of section [3] below) the ambient distribution of the corresponding flag structures 
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are either TdJt or a contact distribution on dJl (i.e. the symbol m is either the commutative 
algebra or the Heisenberg algebra of an appropriate dimension) and the group naturally acting 
on A is either the General Linear group or the symplectic group or the conformal symplectic 
group. All possible symbols 6 of curves of flags with respect to these three groups were listed 
in [171 section 7] and their universal algebraic prolongation u^(5) were calculated in \n\ section 
8], using the representation theory of s[2. In section 3 we describe the u(m,u^((5)) for several 
particular symbols 6 or refer to our previous papers, where they were calculated. The calculations 
of u(m, U''^((5)) for any flag symbol 6 with respect to the three aforementioned groups will be given 
in and [21]. 

Remark 2.4. (the case when additional structures are given on the distribution A) 

Assume that an additional structure is given on the distribution A such that the presence of this 
structure allows one to reduce the bundle P^{xn) to a principle bundle with the structure group 
qO ^ G'O(m). For example, for the flag structures discussed briefly in subsection 11.31 and, in more 
detail, in Example 4 of the next section, we have that A = T9K and the symplectic structure is 
given on 9Jt. In this case m = and P'^(m) coincides with the bundle J-'(9Jt) of all frames on 
Tl. Fixing a symplectic structure on m we can reduce the bundle J^(JM) to the bundle with the 
structure group Sp{m) of the so-called symplectic frames such that the fiber over a point 7 S 9Jt 
consist of all isomorphisms from m to T^QJt, preserving the corresponding symplectic structures. 
Returning to the general situation, let g° be the Lie algebra of G^. Then exactly the same theory 
will work if we will replace everywhere starting from Definition 12.21 the bundle P'^(m) by P", the 
Lie algebra 0°(m) by g, and the group G°(m) by G'^. □ 

Remark 2.5. (the case when ^'^ are parameterized curves) Assume that in the flag struc- 
ture (A, {J'^'l-yeajt) each submanifolds 5'^ of flags is a curve with a distinguished parametrization, 
up to a translation, or shortly a parameterized curve. In order to formulate theorems similar to 
Theorems 12.11 and 12.41 we need only to modify the definition of a symbol of a curve of flags and its 
universal algebraic prolongation to the case of parameterized curves, as was done in \n\ subsec- 
tion 4.5]. The symbol of a parameterized curve is an orbit of an element (and not a line) of gr g"^ 
from gr0'^(m) defined up to the adjoint action of ^1.(111) on the space of degree —1 elements. Fix 
a representative 6 of this orbit. Then the 0-degree algebraic prolongation u^'^^'^((5) of the symbol 
(5 of a parametrized curve should be defined as follows: 

(2.14) u^'P^^(5) := {X G (gr0°(m)), : [X, S] = 0}. 

where, as before, (gr0'^(m))g denotes the space of all degree endomorphisms of the graded space 
gr0~^ (corresponding to the filtration (|1.3p ) belonging to gr0''(m). The spaces u^'^^^{6) for A; > 
are defined recursively, using (|2.8p with u^_^(5) replaced by u^^^'^(5) and the universal algebraic 
prolongation u^'P'^'" of the symbol 5 of a parametrized curve of fiag is defined as 

fc>-l 

Finally, in order to formulate the results analogous to Theorems 12.11 and 12.41 for a flag structure 
(A, {^'^}'y£m) with the constant Tanaka symbol m (of A) and with ^'^ being parametrized curves 
with the constant symbol 6, we need just to replace u^{S) by u^'P^''(5) in these theorems. □ 
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3. Applications: flag structures via linearization 

In this section we describe how the hnearization procedure works in general and show how the 
developed theory can be applied to equivalence problems for various types of differential equa- 
tion, bracket-generating distributions, sub-Riemannian and more general geometric structures 
mentioned in the Introduction. Before going to these examples, let us consider the following gen- 
eral situation. Let 9J{ be a smooth manifold endowed with a tuple of distributions (A,C, {Vi}i£z) 
satisfying the following properties: 

(PI) C and {Vjjigz are subdistributions of A ; 
(P2) Distribution C is integrable and satisfies 

[C, A] c A. 

In other words, any section of the distribution C is an infinitesimal symmetry of the 
distribution A. 

(P3) The distribution Vi is a subdistribution of Vj-i for every i £ Vi + C = V for i < r , 
Vi = Vs for i > s for some integer r and s, and 

(3.1) [C,Vi]CV^-l+C 

(P4) For every integers i spaces Vi{x) C{x) have the same dimensions at all points x € 

We are interested in the (local) equivalence problem for such tuples of distributions (A, C, {Vijigz) 
with respect to the action of the group of diffeomorphisms of M. 

In many cases the tuple of distributions (A,C, {Vijjgz) will satisfy also the following strength- 
ening of property (P3): 

(P3 ') There exists io € Z such that 

(3.2) yi<io [C,V^]=Vi-l+C, 

^. , \ f M r \ there exists a vector field ?) tangent to Vj_i 1 

(3.3) y^>^o V.(x) = |« G H_i(x) : that 0(x) = ^ and [C, ^](x) C V._i(x) /' 

If the property (P3') holds, then all distributions Vi with i > io and all distributions Vi + C 
with i < io are determined by the pair of distributions (C, Vig), using (|3.2p and ()3.3p inductively. 
We will say in this case that the tuple of distributions (A, C, {Vijjgz) is generated modulo C by 
the triple of distributions {A,C,Vif^). 

Finally let us discuss the following strengthening of property (P3'): 

(P3") There exists io € Z such that 

(3.4) Vi<io [C,V^]=V^-l, 

and also ()3.3p holds. 

If property (P3") holds, then the whole tuple of distributions {Vijigz is determined by the pair 
of distributions (C, Vi(,) using ()3.4p and ()3.3p inductively. We will say in this case that the tuple 
of distributions (A, C, {Vijiez) is generated by the triple of distributions (A,C, Vjg). In practice, 
one starts with a triple of distributions (A,C, V) such that C and V are subdistributions of A and 
C satisfies the property (P2) above and then, setting V = Vj^ for some integer io, one can generate 
the tuple {Vi}i£z inductively using (|3.4p and (j3.3p . 
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Let Fol(C) be a foliation of 9Jt by maximal integral submanifolds of the integrable distribution 
C. The main idea to treat the equivalence problem under consideration is to pass to the quotient 
manifold by the foliation Fol(C). Indeed, locally we can assume that there exists a quotient 
manifold 

m = m/Foi{c), 

whose points are leaves of Fol(C). Let $ : 9Jt — ?> OJt be the canonical projection to the quotient 
manifold. Then by the property (2) above the push-forward of the distribution A by defines 
the distribution 

A := $,(A) 

on the quotient manifold 971. 
Fix a leaf 7 of Fol(C). Let 

(3.5) := $*(V.(x))}, X G 7. 

Then we can define the map ^'^ from 7 into the appropriate variety of flag of the space A (7) as 
follows: 

(3.6) a^(x) = {5j(x))},ez, xer, Viez ^/(x) C ^/.^(x) 

The map 2"' or its image in the appropriate flag variety is called the linearization of the sequence 
of distributions {Vi}i(zz along the foliation Fol(C) at the leaf j. This image will be denote by 
as well. 

By means of the linearization procedure we obtain from the original tuple of distributions 
(A,C, {Vi}i£z) the distributions A on the manifold 9Jt with the distinguished submanifolds in 
an appropriate flag variety on each fiber A(7) of A, 7 G SOT, i.e. the flag structure (A, {■^'^j-ygsrit) . 

Note also that 9J{ ^ 9Jt is exactly the tautological bundle over SOT in the sense of Remark 12.31 

It is clear from our constructions that if two tuples of distributions satisfying properties (Pl)- 
(P4), are equivalent, then the corresponding flag structures obtained by the linearization procedure 
are equivalent. The converse is not true in general, but it is obviously true if every distribution 
Vi 7^ contains the distribution C. Namely, we have 

Proposition 3.1. Assume that two tuples of distributions (A, C, {Vijjgz) and (A^, C^, {V/}jgz) 
satisfy properties (P1)-(P4) and, in addition, C is a subdistribution ofVi andC^ is a subdistribution 
ofV} for any i G Z. Then these two structures are equivalent if and only if the corresponding flag 
structures obtained by the linearization procedure are equivalent. 

Note that the application of the linearization procedure to the tuple (A,C, {Vijjgz) and to the 
tuple (A,C, {Vi + C}jgz) leads to the same flag structure. 

Definition 3.1. We say that the tuple (A, C, {Vjjigz); satisfying properties (P1)-(P4), is recov- 
erable by the linearization procedure, if each subdistribution Vi is intrinsically recovered from the 
tuple (A,C,{Vi +C}igz). 

One example of such intrinsic recovery is when Vi is a unique integrable subdistribution of 
maximal rank of Vi + C. Another example is when Vi is the Cauchy characteristic subdistribution 
of Vi-i + C, i.e. the maximal subdistribution of Vj_i + C such that [C, Vj] C Vi-i + C. As an 
immediate consequence of Proposition 13.11 we get the following 
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Corollary 3.1. Two tuples of distributions satisfying properties (P1)-(P4) and recoverable by the 
linearization procedure are equivalent if and only if the corresponding flag structures obtained by 
the linearization procedure are equivalent. 

In the most applications C will be a distribution of rank 1 (a line distribution). In this case, 
after the linearization procedure, one gets the distinguished curves of flags on each fiber of A. 
From relation of property (P2) it follows that the submanifolds of flags 2"' are not arbitrary 
but they are the compatible with respect to differentiation as described in subsection 12.31 following 
the terminology of [T71 Section 3]. 

Remark 3.1. Finally note that if our original structure (A, C, {Vijigz) satisfies the property 
(P3') for some integer ig, then the submanifold of flags = {^J {x)}i^z is completely determined 
by the submanifold in the corresponding Grassmannian. Moreover, = {5j(x)}jg2 is the 
so-called maximal refinement of in the sense of \17\ section 6].n 

As a direct consequence of Theorem 12.41 we have the following 

Theorem 3.1. Assume that tuples of distributions (A, C, {Vjjjgz) satisfy conditions (P1)-(P4) 
and the resulting flag structures (A, {^'^l^ggjt) satisfy conditions (F1)-(F4) with the constant 
Tanaka symbol m (of A) and with the constant flag symbol 6. Assume also that the universal 
algebraic prolongation u(m,u^((5)) of the pair (m,u^((5)) is finite dimensional. Then to any such 

tuple (A, C, {Vijigz) one can assign a bundle overW of the dimension dim u(m, u^(5)) endowed 
with the canonical frame. If in addition, the tuples of distributions (A, C, {Vj}.jgz) ore recoverable 
by the linearization procedure, two of them are equivalent if and only if the canonical frames 
assigned to them are equivalent. 

This theorem is of a quite general nature and, as the examples below show, it treats in a 
unified way many previously known results and generalizes them significantly. Using this theorem 
one can obtain the main results of [S] and [TU] on contact geometry of scalar ODEs of order 3 
and of order greater than 3 , respectively, (see Example 1 below) and generalize them to various 
equivalence problems for systems of differential equations, including the systems with different 
highest order of derivatives for different unknown functions, called the systems of ODEs of mixed 
order (see Example 2 below). Using Theorem 13.11 one can obtain the main result of |14j on 
canonical frames for rank 2 distributions on manifolds of arbitrary dimension. Our approach 
here also gives much more conceptual point of view on the construction of the preprint [16] on 
rank 3 distributions, or, more precisely, the part of these results addressing the existence of the 
canonical frames. It generalized the results of [141 [T6] to distributions of arbitrary rank (see 
Example 3). The modification of this Theorem, corresponding to the flag structure with ^'^ 
being parameterized curves of flags gives the unified construction of canonical frames for sub- 
Riemannian, sub-Finslerian, and more general additional structures on distributions. 

In the context of Theorem 13.11 the following natural question arises: For what pairs (5, m) is 
the canonical frame of Theorem 13. II can be chosen as the Cartan connection over SOT? The answer 
to this question, even in the case of the commutative or Heisenberg algebra m, is known yet for 
several particular cases only. 

Example 1. Scalar ordinary difTerential equations up to a contact transformation. 

Consider scalar ordinary differential equations of the form 



(3.7) 



y(-) = /(x,y,y',...,y(-i)). 
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Each such equation can be considered as a hypersurface 8 in the jet space J"(M, M), caUed 
the equation manifold. Equations ()3.7p are solved with respect to the highest derivative, so 
the restriction of the natural projection vro: J"(M, M) — )> J"~^(R, M) to the hypersurface £" is a 
diffeomorphism. Two such equations 8 and 8' are said to be contact equivalent, if there exists 
a contact transformation J^(M,M) J^(M, M) with a prolongation to J"(]R,M) such that 
$"(^) = 8'. 

Note that by Lie-Backlund theorem any diffeomorphism of J"(IR, M) preserving the Cartan 
distribution (i.e. the rank 2 distribution defined by the contact forms 9i = dyi — yi+idx, i = 
0, . . . , n — 1, in the standard coordinates (x, yo, . . . , yn)) is the prolongation to J"(M, M) of some 
contact transformation of J^(M, M). Therefore two equations 8 and 8' are contact equivalent if 
and only if there is a diffeomorphism ^ of J"'(M,M) which preserves the Cartan distribution on 
J"(M,M) and such that ^{8) = 8'. 

The equivalence problem for ordinary differential equations up to contact transformations is a 
classical subject going back to the works of Lie, Tresse, Elie Cartan [8], Chern [9], and others. 
The existence of the normal Cartan connection (a special type of frames) associated with any 
system of ODEs was proved in [9] for the equations of third order and in [10] for n > 3, The latter 
is based on the Tanaka-Morimoto theory of normal Cartan connection for filtered structures on 
manifold [Ml IMl ES] . 

The reason we start with this example is because the problem of contact equivalence of equations 
()3.7p can be reformulated as the equivalence problem for a very particular tuples of distributions 
satisfying properties (P1)-(P4) and the linearization procedure in this case gives a shorter way to 
obtained important contact invariant of ODE's, the so-called generalized Wilczynski invariants 
[llj . without constructing the whole normal Cartan connection mentioned above. It also gives an 
alternative way for the construction of the canonical frame in this problem. 

The Cartan distribution on J"'(R, R) defines the line distribution X on 8. This distribution is 
obtained by the intersection of the Cartan distribution with the tangent space to 8 at every point 
of 8. Note that the corresponding foliation Fo^/S) is nothing but the foliation of the prolongations 
of the solutions of our ODE to J"(]R,M). In the coordinates {x,yo, . . . ,yn-i) on 8: 



Further, let VTj: J"(M,M) H> J" ^ *(]R, M) be the canonical projection, < i < n — I. For any 
e £ 8 we can define the filtration {Vi{e)}^^Q of T^f as follows: 



Then is a rank i distribution on In the coordinates {x,yQ, . . . , yn-i) on 8 we have 



The tuple of distributions (A,C, {Vi}i^z), where A = T8, C = X , Vi = for i > 0, Vi = V-i for 
— n + 2 <i < —1, V-n+i = Vn-i + and Vi = T8 for i < —n + 1, satisfies properties (P1)-(P4) 
above. We say that this tuple of distributions is associated with the ordinary differential equation 
(j3.7|) with respect to the contact equivalence Below we justify this terminology. Note that relations 
(j3.8|) and (j3.1Up imply that it satisfies the property (P3') for any io satisfying — n + 2 < i < — 1 
and even the property (P3") for = n — 2. The latter implies that two ordinary differential 
equations are contact equivalent if and only if the tuples of distributions associated with these 
equations are equivalent. 



(3.8) 




(3.9) 



Vi{e) = keTd,TTinTe8. 



(3.10) 
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Moreover, the constructed tuple (A, C, {Vijigz) is recoverable by the linearization procedure. 
For this we only need to check that the distributions Vj with —n + 2 < i < —1 can be intrinsically 
recovered from the tuple (A, C, {Vi©C}i(=z). The latter follows from the fact that Vj is the Cauchy 
characteristic subdistribution of Vi_i +C, which is a part of the proof of the Lie-Backlund theorem 
in this case. 

Let Sol denote the quotient manifold £/Fol{X), i.e. the manifold of the prolonged solutions of 
the equation manifold £. Consider the linearization ^'^ = {d'i}iez of the sequence of distributions 
{Vi} along the foliation Fol(A') at a leaf 7 (i.e. along a prolongation of equation (|3.7|) ). By our 
constructions, dim ^]{s) = —i if — n + I < i < —1, dim ^]{e) = if i > 0, and dim ^]{e) = n — 1 
if i < — n + 1. In particular, is a curve in the projective space PT^Sol. 

Moreover, relations (|3.8|) . (|3.1U|) . and Remark [XT] imply again that for any e € 7 the space •37(e) 
of r^Sol is exactly the i-th osculating space of the curve at e. In other words, the whole curve of 
flags ^'^ is completely determined by the curve -31^ in a projective space. Note also that a curve in a 
projective space corresponds to a linear differential equation, up to transformations of independent 
and dependent variables preserving the linearity. For the curve -31^ this linear equation is exactly 
the classical linearization (or the equation in variations) of the original differential equation (j3.7p 
along the (prolonged) solution 7. This is why we use the word linearization here. 

The symbol 5^ of the curve of complete flags (with respect to GL(TySol)) at any point is a line 
of degree —1 endomorphisms of the corresponding graded spaces, generated by an endomorphism 
which has the matrix equal to a Jordan nilpotent block in some basis. The flat curve with this 
symbol is the curve of osculating subspaces of a rational normal curve in the projective space 
PT^Sol. Recall that a rational normal curve in ¥W is a curve represented as i 1— )• [1 : t : : . . . t"] 
in some homogeneous coordinates. It is well-known that the algebra of inflnitesimal symmetries 
of a rational normal curve and, therefore, the universal algebraic prolongation u^{S^) is equal to 
the image of the irreducible embedding of gl2(K) into ^[(TySol) (see also [T71 Theorem 8.3], where 
the universal algebraic prolongation was computed for all symbols of curves of flags with respect 
to the General Linear group). 

Further, recall that in this case the symbol of the distribution A is isomorphic to the n- 
dimensional commutative Lie algebra, i.e. to M". By direct computations, the Tanaka universal 
algebraic prolongation u(M", u'^((5^)) of the pair (M", u'^((5^)) is isomorphic to sp4(M), if n = 3, 
and to a semidirect sum of 0l2(I^) ^ of 312(1^) and M", if n > 3 (were 0t2(IR) acts irreducibly 
on M"), as expected from [9] in the flrst case and from [10] in the second case. Note also that 
in the second case already the first algebraic prolongation u^(M",u^((5^)) is equal to zero, i.e. 
the canonical bundle lives already on the quasi-principle bundle P assigned to the flag structure 
(A, {^^}'y£<xil) by Theorem 12. II Note also that the local contact geometry of second-order ODEs is 
trivial, since any two such equations are locally contact equivalent and the corresponding algebra 
{S2)) is infinite dimensional with the kth. algebraic prolongation equal to the space if 
M^-values homogeneous polynomials of degree k + 1. 

Further, it follows from \18\ subsection 4.1] that the symbol (5^ is nice in the sense of Remark 
12. 3i In this way the classical fundamental system of invariants of curves in projective spaces, 
the Wilczynski invariants, can be constructed (see |17] or [TT] for the detail) for each curve ^'^ 
and they produce the contact invariants of the original ODE, called the generalized Wilczinski 
invariants (|llj). 

Finally, it can be shown that the normalization condition for Theorem 12.41 can be chosen such 
that the resulting canonical frame will be a Cartan connection over the equation manifold £ 
modeled by the corresponding 5p4(M) parabolic geometry if ?z = 3 and by a homogeneous space 
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corresponding to a pair of Lie algebras 0l2(M) XM" and t2(M) if n > 3, where t2(M) is the algebra 
of the upper triangle 2 x 2-matrices. 

Example 2. Natural equivalence problems for systems of two differential equations 
of mixed order. Given two natural numbers k and / and an integer s satisfying < s < / 
consider the following system of two differential equations: 

in / y^^^ = ^1 ?/(^)' ■ ■ • ' y^*^"'^ {X),Z{X),Z'{X), z{mm{i-l,i-s}) (^)) 

^ ' I z('\x) = h{x,y{x),y'ix),...,y^'^-^\x),z{x),z'{x),...,z(^-^\x)) 

Such system will be called a system of differential equations of mixed order {k,l) with shift s. 
Obviously, systems having a shift s have a shift s — 1 as well. System (|3.11|) defines a submanifold 
So in the mixed jet space j('^'')(R, M^): in the standard coordinates {x,yo,yi, ■ ■ ■ ,yk, zo, zi, . . . , zi) 
in R^) the submanifold Sq has a form: 

(•3-^2) / = fii^^yo^yi^ ■ ■ ■ ^yk-i,z,zi, . . . ,z,^ia{i-i,i-s}) 

\ zi = f2{x,yo,yi,...,yk-i,z,zi,...,zi_i) 

Differentiate both parts of the first equation of the system p. lip with respect to the inde- 
pendent variable x and replace z^''\ if it appears in the obtained equation, by the right hand 
side of the second equation of p. lip . With this new equation, written in the first place, we 
obtain a new system of three equations. We call this system of equation the first prolongation 
of the system (|3.1ip with respect to the first equation. This new system defines a submanifold 
£1 in the mixed jet space j('^"'"^'')(M, M^): to obtain this submanifold in the standard coordinates 
{x,yo,yi, . . . ,yk+i,zo,zi, . . . ,zi) in J(''+^'')(]R,M2) we replace by yi and z^-?') by Zj in the new 
system obtain. The submanifold f 1 will be called the first prolongation of the submanifold £q with 
respect to the first equation of the system (j3.1ip . 

Similarly, differentiating the first equation of the new system once more and replacing z^^\ 
if it appears in the obtained equation, by the right hand side of the second equation of (j3.1ip . 
we obtain, together with already existing equation, the system of 4 equations, called the second 
prolongation of the system ()3.1ip with respect to the first equation and the submanifold £2 in the 
mixed jet space J'''^"'"^'^'*(M, M^), called the second prolongation of the submanifold £q with respect 
to the first equation of the system ()3.1ip . In the same way, by induction for any s > we construct 
the system of s + 2 equations and the submanifold £s in the mixed jet space 

J(*^+^'0(M,m2), which 

will be called the prolongation of order s of the system ()3.1ip and of the submanifold £q with 
respect to the first equation of the system ()3.1ip . 

Further, the Cartan distribution is defined on any mixed jet space J(^''")(M,M2). In standard 
coordinates {x, yo, . . .yj,, zq, . . . , zj) in jC^'') (M, M?) the Cartan distribution is defined by the fol- 
lowing system of Pfaffian equations: 

dyi - yi+idx, z = 0, . . . , A: - 1; 
dzj — Zj+idx,j = 0, — 1. 

Definition 3.2. Consider two systems of differential equations of mixed order {k,l) with shift s. 
Assume that £q and £q are the corresponding submanifolds in the mixed jet space ana 
£s and £s are the prolongations of order s of the submanifold £q and £q with respect to the first 
equation of the corresponding systems. We say that our systems ( or submanifolds £q and £0 ) are 
s-equivalent (with respect to the first equation) if 
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(1) in the case < s < I — k there exists a diffeomorphism $ of the mixed jet space 
j(o,z-fc-s)^]^^ ]^2-j preserving the Cartan distribution on it such that the prolongation of 
<1> to the mixed jet space j(fc+^.0(M^]R2) sends £s onto Ss- 

(2) in the case l—k < s < I there exists a diffeomorphism ^ of the mixed jet space 
preserving the Cartan distribution on it such that the prolongation of $ to the mixed jet 
space 

j(fc+s,0(]^^]R2) sends £s onto £s- 

Note that given si and S2 with < si < S2 < ^ the si-equivalence and the S2-equivalence are in 
general two different equivalence problems on the set of systems of mixed type (fc, I) with shift S2 
(which is the common set of objects for which both equivalence problems are defined). Symmetries 
of the trivial system of the mixed order (/c, Z), namely of the system = 0, z^'^ = 0, with respect 
to the s-equivalence are calculated in the recent preprint |19] . In particular, it was shown that 
in the case (/c, I) = (2, 3) the groups of symmetries for s = and s = 1 are both 15-dimensional 
but not isomorphic one to another. By symmetries with respect to the s-equivalence of a system 
of mixed order [k, I) corresponding to a submanifold £q of the mixed jet space IR we 

mean any diffeomorphism ^ of either the mixed jet space J(°'^-'=-*)(M,M2) or the mixed jet space 
J(^+"-''0)(M,M2) (depending on the sign oil — k — s) such that the prolongation of to the mixed 
jet space j('^"'''''')(M, R^) preserves the prolongations £s of order s of £q with respect to the first 
equation of the systems. Note also that if s = / — > 0, the s-equivalence is nothing but the 
equivalence of the s-prolongation £s of £q with respect to the first equation of the system up to 
point transformations. 

Now let us show that the introduced s-equivalence is a particular case of the equivalence prob- 
lems introduced in the beginning of this section. First, system (|3.1ip defines a rank 1 distribution 
■^k,i,s on £s: its leaves are prolongations of the solutions (y(x),z(x)) to the mixed jet space 
can be equivalently defined as the rank 1 distribution obtained by the intersec- 
tion of the Cartan distribution on j('^"''*'^)(]R, M^) with the tangent space to £s at every point of 
£s. 

Let TTs^i-. J(''+'''')(M,M2) J{'^-+^-i-«'^-i-»)(]R^m2) be the canonical projections , and let 
(3.13) Vs,i{e) = ker 47r,,i n T.f,. 

Here < i < minj/c + s — 1, / — 1}. In the coordinates (x, yo, . . . , yk-i,zo, . . . , zi^i) on £s we have 
Vso = and 



(3.14) Vs, 




l<i< s 



{4tW s<i<mm{k + s-l,l-l} 



j=k—i+s^ dzj ) j 



We also assume that Vg^i = for i > 0, while for i > min{A; + s — 1, / — 1} we define Vg^i inductively 

by Vs,i = [Xk,i,s, Vs,i-i]. Then the tuple of distributions (A,C, {Vjjgz) with A = T£s, C = Xk,i,s, 
and Vi = Vs^-i satisfies properties (P1)-(P4) above. We say that this tuple of distribution is 
associated with the system of differential equation of mixed order ()3.7p with respect to the s- 
equivalence. Using (|3.14p . it is not hard to show that it satisfies the property (P3') for any io 
satisfying s < io < min{/c + s — 1, / — 1} and even the property (P3") for ig = min{A; + s — 1, 1 — 1}. 
The latter implies that two systems of mixed order {k, I) with shift s are s-equivalent if and only 
if the tuples of distributions associated with these equations are equivalent. 

From (j3.14p it follows easily that the curves of flags which are the linearizations of the 
sequence of distributions {Vi}i^z along the foliation Fol{Xi^^i^s) along leaves 7, have the same 
symbol at any point. Let us describe this symbol. For this let us introduce some notation. 
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following [n\ subsection 7.1]. Let 6i and 82 be degree —1 endomorphisms of the graded spaces 
Vi and V2, respectively. The direct sum Vi © V2 is equipped with the natural grading such that 
its ith component is the direct sum of ith. components of Vi and V2. The direct sum 5i 82 is 
the degree — 1 endomorphism of Vi © V2 such that the restriction of it to Vi is equal to 6i for each 
i = 1,2. A degree —1 endomorphism 5 of a graded space V is called indecomposable if it cannot 

be represented as a direct sum of two degree —1 endomorphisms acting on nonzero graded spaces. 

p 

Further, given two integers r < p < let Vrp = ^^Ei, where dim Ei = 1 for every i, r < i < p, 

i=r 

and let 6rp be the degree —1 endomorphism of Vrp which sends Ei onto Ei-i for every i, r < i < p, 
and sends Ej. to 0. For example in this notation the symbol (5^ appearing in the previous example 
is equal to M(5_„^_i. As was shown in [T71 subsection 7.1] the endomorphisms 5rs are the only, 
up to a conjugation, indecomposable degree —1 endomorphism of a graded space of curve of flags 
with respect to the General Linear group. In other words, any degree —1 endomorphism of a 
graded space is a direct sum of endomorphisms of type 6rp- In a similar way one define the direct 
sum for symbols and the notion of indecomposable symbol with respect to the General Linear 
group. 

Using (|3.14p . it is easy to see that the symbol in the current example is isomorphic to the line 
generated by 

(5„i „i 6_s-k-s-i- 

The universal algebraic prolongations of all possible symbols of curves of flags with respect to 
the General Linear group was calculated in jl7|. sections. 2] using the representation theory of 

s[2. The Tanaka universal algebraic prolongation u^M*^^^, it^(M(5_/^_i @S-s-k~s~i))^ of the pair 

u^(R((5_;^_i 5_s-fe,-s-i)) is calculated in [19], using the fact that it is isomorphic to the 
algebra of infinitesimal symmetries of the trivial system of the mixed order {k,l), namely of the 
system = 0, z^'-* = 0, with respect to the s-equivalence. In |18l section 4.1] it was shown that 
the symbol of curves of flags in the case k = 2, I = 3, and s = are not nice. We expect that this 
is the case also for all symbols with k < I and s > 0. 

One can consider more general systems of m differential equations of mixed order and m — 1 
shifts (or, shortly, a given multishift) such that this multishift has some natural restrictions. Then 
one can define a natural equivalence relation for such systems according to this multishift. Passing 
to the corresponding flag structures, one obtains that the curves of flags have the symbol 6 which 
is a direct sum of m indecomposable symbols of the type. Such equivalence problems and the 
corresponding Tanaka universal prolongations from Theorem 13.11 will be described in [20j . 

Example 3. Geometry of distribution via abnormal extremals. Consider the problem 
of equivalence of bracket generating distributions of a given rank with respect to the action of 
the group of diffeomorphisms of the ambient manifold. The classical approach to this problem is 
provided by Tanaka theory [6l [331 HI] described shortly in subsection 12.41 above. Assume that D 
is a distribution with constant Tanaka symbol t . Consider the corresponding bundle P^{i) (as 
defined in subsection 11.71 above). Then the construction of a canonical frame for D is given by 
Theorem [221 (with m = t and 3° = g°(t)). 

However, in order to apply the Tanaka machinery to all bracket-generating distributions of the 
given rank / on a manifold of the given dimension n, one has to classify all n-dimensional graded 
nilpotent Lie algebra with / generators and also one has to generalize the Tanaka prolongation 
procedure to distributions with non-constant symbol, because the set of all possible symbols may 
contain moduli (may depend on continuous parameters). Note that the classification of all symbols 
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(graded nilpotent Lie algebras) is a quite nontrivial problem already for n = 7 (see [25]) and it 
looks completely hopeless for arbitrary dimensions. 

In a series of papers \13\ [T3] and the preprint [16j we proposed an alternative approach which 
allows to avoid a classification of the Tanaka symbols. It is based on the ideas of the geometric 
control theory and leads, after a symplectification of the problem, to the equivalence problems 
for a particular class of structures discussed in the beginning of this section. By symplectification 
we mean the lifting of the original distribution to the cotangent bundle. 

In more detail, let D he a distribution on a manifold M. First, under some natural generic 
assumptions we distinguish a characteristic 1- foliation (the foliation of abnormal extremals) on 
a special odd-dimensional submanifold of the cotangent bundle associated with D 

For this first introduce some notations. Taking Lie brackets of vector fields tangent to a 
distribution D (i.e. sections of D) one can define a filtration C C ... of the tangent 
bundle, called a weak derived flag or a small flag (of D). More precisely, set = D and 

define recursively = + [D, D~^^^]. We assume that all are subbundles of the tangent 
bundle. Denote by (D^)-^ C T*M the annihilator of , namely 

(Z)jy = {{p, q) G T*M \ p-v = G D^{q)}. 

Let TT : T*M i-^ M be the canonical projection. For any A G T*M, X = ip,q), q G M, p £ T*M, let 
*'(-^)(") — Pi"^*') t>e the canonical Liouville form and a = be the standard symplectic structure 
on T*M. 

The crucial notion in the symplectification procedure of distributions is the notion of an ab- 
normal extremal. An unparametrized Lipshitzian curve in D-^ is called abnormal extremal of a 
distribution D if the tangent line to it at almost every point belongs to the kernel of the restriction 
ctI^x of a to -D"*- at this point. The term "abnormal extremals" comes from Optimal Control 
Theory: abnormal extremals of D are exactly Pontryagin extremals with zero Lagrange multiplier 
near the functional for any variational problem with constrains, given by the distribution D. 

Now let us describe the submanifold of T*M foliated by the abnormal extremals. First set 

(3.15) Wd-={\(^D^ ■.\ier{a\D^{\)) ^{)} 
Then 

(1) If rankD is odd, then Wd = D^, because a skew-symmetric form in an odd dimensional 
vector space has nontrivial kernel; 

(2) If rank!) = 2, then it is easy to show [Ml Proposition 2.2] that Wd = {D'"^)^; 

(3) More generally, if rank!) = 2k then the intersection of Wd with the fiber D-^{q) of 
over a point g G M is a zero level set of a certain homogeneous polynomial of degree k on 
D-^{q). This polynomial at a point A = {p,q) G D-^ is the Pfaffian of the so-called Goh 
matrix at A: if Xi, . . . is a local basis of the distribution D, then the Goh matrix at A 
(w.r.t. this basis) is the matrix (p • [Xj, Xj](q))^^.^^. 

In the sequel, for simplicity, we will assume that rankZ) is odd or rank!) = 2. In both cases 
Wd is an odd dimension submanifold. Therefore ker(o"|p^;^(A)) is nontrivial. Define a subset Wd 

of Wd as follows: 

(3.16) Wd ■= {A G Wd ■ ker((T|^^(A)) is one-dimensional}. 

By direct calculation one can show that 

(1) If rankZ) = 2, then Wd = {D~'^)^\{D^^)^ ([Ml section 2]); 
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(2) If rank!) = 3, then Wd = D^\{D~^)-^ ([l6]). 
Consequently, for any bracket generating rank 2 or rank 3 distribution on a manifold M with 
dimM > 4 the set Wd is an open and dense subset of Wd- In the sequel we will assume that the 
set Wd is an open and dense subset of Wd- Note that this is a generic assumption for distributions 
of odd rank greater than 3. See also Remark 13.21 below addressing the case when this assumptions 
does not hold. 

By constructions, the kernels of (tIvi/d form the characteristic rank 1 distribution A on Wd- 
The integral curves of this distribution are abnormal extremals of the distribution D. Note that 
in general, these are not all abnormal extremals of D, however for our purposes it is enough to 
work with these abnormal extremals only. 

Further, it is more convenient to work with the projectivization of PT*M rather than with 
T*M. Here PT*M is the fiber bundle over M with the fibers that are the projectivizations of the 
fibers of T*M. The canonical projection 11 : T*M — >■ PT*M sends the characteristic distribution A 
on Wd to the line distribution A on ¥Wd {'-= ^(Wd)), which will be also called the characteristic 
distribution of the latter manifold. 

Further note that the corank 1 distribution on T*M annihilating the tautological Liouville form 
<; on T*M induces a contact distribution on PT*M, which in turns induces the even-contact (quasi- 
contact) distribution A on F{D'^)^\¥{D^)^ . The characteristic line distribution A is exactly the 
Cauchy characteristic distribution of A, i.e. it is the maximal subdistribution of A such that 

(3.17) [A,A]cA. 

Further, let vf : ¥T*M i— >■ M be the canonical projection. Let J be the puUback of the original 
distribution D to ¥Wd by the canonical projection fr: 

(3.18) ^(A) = {ve TxFWd ■- € Dm\))} 
and V be the tangent space to the fibers of Wd 

(3.19) V{X) = {ve TxFWd ■- n^v = 0}. 

Note that V + A C J'. We work with the distributions A, V, and J instead of the original 
distribution D. 

Now define a sequence of subspaces c7*(A), A G PVFd, by the following recursive formulas for 
i < 0: 

(3.20) J',_i(A) := [A, Ji\{X), J_i(A) = J{X) 
Directly from the fact that ^ is a line distribution it follows that 

(3.21) dim JL2(A) - J-i{X) < rank!) - 1. 

From (|3.17p it follows that Ji d IS. for alH < 0. Note that the symplectic form a induces the 
antisymmetric form a on each subspace of a distribution A, defined up to a multiplication by a 
constant, and A is exactly the distribution of kernels of this form. 

Given a subspace A of A(A) denote by the skew-symmetric complement of W with respect 
to this form. It is easy to show that the spaces ^(A) are coisotropic with respect to the form 
a, i.e. they contain their skew symmetric complement: i7(A)^ C J^{X). Moreover, if rankD = 2 
then in fact i7(A)^ = J{X)- Using the operation of skew-symmetric complement we can define 
the subspaces Ji{X) for z > as follows 
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(3.22) 



(j„i_i(A))^nF(A), rank!) = 2, 
{J~i~2{X)Y n V{\), rank!) is odd . 



Note also that if rankD is odd, then = V{X) + ^(A), which imphes that in this case 



Similarly, if rank!) = 2 and dimL>2(g) = 3, then J^2{>^Y = VW + A{\) for any A G ¥Wd 
with vr(A) = q, so formula (j3.23p holds in this case as well. Besides, it is easy to see that 
[A^ Ji] C Ji^i + A also for nonnegative i. 

Further, for a generic point q € M there is a neighborhood U and an open and dense subset U 
of 7r~^(C/) n FWd such that for any i G Z dim j7i(A) is the same for all X G U. Then the tuple of 
distributions (A,(J, {Vjjigz) on U with C = A and Vj = J7i satisfies properties (P1)-(P4) above. 
We will say that this tuple is associated with the distribution D by the symplectification. From our 
constructions and formula (|3.23p it follows immediately that two distributions are equivalent if 
and only if the corresponding tuples of distributions associated by symplectification are equivalent. 

Moreover, in most of the situations V is an integral subdistribution of y + ^ of maximal rank. 
The latter condition implies that the tuple {A,A,{jTi}i^z) is recoverable by the linearization 
procedure. The linearization of the sequence of distributions {Ji}i^z along the foliation Fol(^) 
of abnormal extremals at a leaf (an abnormal extremal) 7 is called the Jacobi curve of the abnormal 
extremal 7. From ()3.20p . ()3.22p . and Remark l2.1l it follows that the curves of flags satisfy conditions 
(F1)-(F3) of subsection [231 

Let 5 be a symbol of a curve of symplectic flags or, shortly, a symplectic symbol. A point 
A G FWd is called 6- regular if there is a neighborhood [/ of A in ¥Wd such that for any A G C/ 
if 7 is the abnormal extremal passing through A, then 6 is the symbol of the curve at A. Note 
that by constructions the line distribution A depends algebraically on the fibers. I turns out (see 
|21| for detail) that from this, the fact that the set of all symplectic symbols of Jacobi curves is 
discrete and from the classification of these symbols given in jl71 subsection 7.2] it follows that 
for distributions of rank 2 or of odd rank (and for distributions of any rank if we work over C) 
for a generic point q € M there is a neighborhood U oi q in M and a symplectic symbol 5 such 
that any point from a generic subset of tt~^{U) nPW/) is 5-regular. Moreover for a generic point 
q GU the set of all (5-regular points in TT~^{q) n PVFd is Zariski open. We call the symbol 5 the 
Jacobi symbol of the distribution D at the point q 

Now the problem is to construct the canonical frames uniformly for all distributions with given 
Jacobi symbol 5. If we apply the linearization procedure to the tuple (A, {jTijigz) on the set 
7r~^{U) n FWd then the resulting flag structure (A, {^'^}'y^<m) has the constant flag symbol 6 at 
generic points of the curve ^'^ of a generic abnormal extremal 7. Despite this property is weaker 
than the constancy of the flag symbol, the conclusion of Theorem 12 . 41 still holds true if we restrict 
ourselves to the points of the curves ^'^ , where the flag symbol is isomorphic to 5. Recall also 
that the symbol of A in this case is isomorphic to the Heisenberg algebra n of the appropriate 
dimension with the grading © g~^, where is the center. Therefore, by Theorem 13.11 the 
construction of the canonical frame for distributions with given Jacobi symbol 6 is reduced to the 
calculation of the algebra u(n, u^{6)) . A natural generic subclass of distributions are distributions 
of the so-called maximal class |13j-|16j. A distribution is called of maximal class if all curves 
do not lie in a proper subspace of A (7) for generic abnormal extremal 7. Obviously this property 
is encoded in the Jacobi symbol. 



(3.23) 



MX) = V{X). 
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Remark 3.2. The scheme described above will work also for distributions for which the set 
Wd is empty. Indeed, in this case instead of Wd one can consider a subset of Wd for which 
dim ker((T|^ (A) attains its infimum. Clearly it is an open set. Restricting on this set we still 
have an integrable distribution A of this kernels (of rank greater than 1) and we can proceed with 
the linearization procedure as well, getting as submanifolds of dimension > 1 instead of course. 
Although, in this case the assumption of constancy of symbol is not automatical, Theorem 13.11 
still can be applied under this constancy of symbol assumption. □ 

(a) The case of rank 2 distributions. Let us describe the algebra u(n,u^((5)) for rank 2 
distributions of the maximal class. Using ()3.2ip it is easy to show that in the case of rank 2 
distributions the condition of the maximality of class is equivalent to the fact that the flag ■3'^ (A) 
is a complete flag for a generic point A on the curve 7 [Hj. If we let dimM = n, n > 4, the 
latter is also equivalent to the fact that dim^^_^{X) = 1 for a generic point A on the curve 7. 
Moreover, the whole curve at generic points can be completely recovered by osculations by the 
curve 5^_5 in the corresponding projective space (of the dimension 2n — 7) 

From this prospective, the equivalence problem for rank 2 distributions is similar to the contact 
equivalence of scalar ordinary differential equations. The only difference is that for distributions 
there is an underlined (conformal) symplectic structure on A(7). In particular, the curves 5^„5 
are not arbitrary curves in the projective space of A(7), but they satisfy the following property: 
the curve of complete flags obtained from them by the osculation must consist of symplectic flags. 
Such curves in a projective space are also called self-dual [15]. 

The important point is that for a given n there exists the unique Jacobi symbol of rank 2 
distributions of maximal class. To describe it given a natural m let 

(3.24) Cm= 

—m—l<i<m—2 

be a graded spaces endowed with a symplectic form w, defined up to a multiplication by a nonzero 
constant, such that dim = 1 for every admissible i and Ei is skew orthogonal to Ej for all 

1 + j^ —3. Let Tm be a degree —1 endomorphism of Cm from the symplectic algebra which sends 
Ei onto Sj-i for every admissible i, except i = —m — 1, and Tm{E-m-i) = 0. We also assume 
that a;(rm(e), e) > for all e € E^i. Then, by our constructions, for a given n the unique Jacobi 
symbol of rank 2 distributions of maximal class is isomorphic to the line generated by Tns- 

Disregarding the underlying conformal symplectic structure on q~^, and up to a shift in the 
chosen weight of the grading, this Jacobi symbol is the same as the symbol of a scalar ordinary 
differential equation of order 2n — 6, i.e. 52n-6 ™ notations of Example 1. Note that in the 
notation of Example 2 and again disregarding the underlying conformal symplectic structure it is 
exactly R52-n,n-5- 

It is not clear yet if the assumption of maximality of class is restrictive. We checked by direct 
computations that for n < 8 all bracket generating rank 2 distributions with small growth vector 
(2, 3, 5, . . .) are of maximal class. Actually we do not have any example of bracket generating rank 

2 distributions with small growth vector (2, 3, 5, . . .) which are not of maximal class. Comparing 
this to the set of Tanaka symbols, for rank 2 distributions with five dimensional cube if n = 6 
there are 3 non-isomorphic Tanaka symbol, if n = 7 there are 8 non-isomorphic Tanaka symbols, 
and the continuous parameters in the set of all Tanaka symbols appear starting from dimension 
8. Since by above, all such distributions with given n, at least for n < 8, are of maximal class 
and therefore have the same Jacobi symbols, it already shows that starting with the Jacobi symbol 
instead of Tanaka symbols as a basic characteristic of rank 2 distributions, we get much more 
uniform construction of the canonical frames. 
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Further, similar to the case of scalar ordinary differential equations (of order 2n — 6), the 
corresponding flat curve is a curve of complete flags, consisting of all osculating subspaces of 
the rational normal curve in a (2n — 7)-dimensional projective space and the universal algebraic 
prolongation u^(r„_3) of the Jacobi symbol of r„_3 is equal to the image of the irreducible 
embedding of gl(2,M) into csp(g~^) (after identifying the graded symplectic spaces and Cn-s)- 
The Tanaka universal prolongation u(n, u(r„_3)) of the pair (n, u(r„,_3)) is equal to the split real 
form of the exceptional simple Lie algebra G2 for n = 5, as expected from the classical Cartan 
work [7], and to the semidirect sum g[(2,]R) X n for n > 5 , i.e. already the first algebraic 
prolongation u(n,u(r„_3)) vanishes in this case, as expected from [l'S\ I14j. Here in the semidirect 
sum of 012(1^) and n = the algebra 0f2(IR) acts irreducibly on Note that in the case 
n = 4 the local equivalence problem for generic bracket generating rank 2 distribution is trivial: 
any two generic germ of rank 2 distribution in are equivalent (the Engel normal form) . In this 
case the algebra n®u(ri) is infinite dimensional and its completion is isomorphic to the algebra 
of formal Taylor series of contact transformations of the jet space J-^(M,]R). 

Further, similarly to the case of ordinary differential equations the Jacobi symbol r„_i is nice in 
the sense of Remark 1 2. 31 In this way, as in Example 1, the Wilczynski invariants can be computed 
for each curve ^'^ and they produce the invariants of the original rank 2 distribution, that by anal- 
ogy with [11] can be called the generalized Wilczinski invariants of the rank 2 distribution. Note 
that the self-duality of the curves in a projective space generating the Jacobi curves implies that 
some Wilczinski invariants, namely the Wilczinski invariants of odd order, vanish automatically. 
The minimal order of possibly nonzero Wilczinski invariants is equal to 4. As shown in [40], in the 
case n = 5 the Wilczinski invariants of order 4 (which is the only invariant of the Jacobi curves in 
this case) coincides with the so-called fundamental tensor of rank 2 distributions in 5 dimensional 
manifold discovered by E. Cartan [7]. 

Finally, it can be shown that the normalization condition for Theorem 12.41 can be chosen such 
that the resulting canonical frame will be a Cartan connection over Wd (= {D~'^)-^\{D^^)-^) 
modeled by the corresponding G2 parabolic geometry in the case of n = 5 and by a homogeneous 
space corresponding to a pair of Lie algebras 0l2(K) X n and t2(M) if n > 5, where t2(M) is the 
algebra of the upper triangle 2 x 2-matrices. 

(b) The case of rank 3 distributions. Using (|3.21|) and disregarding for a moment the 
underlying conformal symplectic structure on g"^ it is easy to show that the Jacobi symbol of a 
rank 3 distribution of maximal class with 6-dimensional square is a direct sum of two indecompos- 
able symbols with respect to the General Linear group (see [l<j\ section 2] where it is formulated 
in different terms using Young diagrams instead of Jacobi symbols). From this prospective, the 
equivalence problem for rank 3 distributions is similar to the s-equivalence of systems of two 
ordinary differential equation of mixed order for some shift s. 

In the case of rank 3 distributions with Jacobi symbol 6 the structure of the Lie algebras u^{S) 
and u(n, U'^((5)) are much more complicated. In contrast to the case of rank 2 distribution, here 
the presence of the additional conformal symplectic structure is already important on the level of 
algebraic prolongation of the flag symbol. First a Jacobi symbol 6 being decomposable with respect 
the General Linear group is symplectically indecomposable in the sense of \17\ subsection 7.2] (it 
is a indecomposable symbol of the type (Dl) there). Second the universal algebraic universal 
prolongation u^{S) with respect to conformal symplectic group is different (strictly smaller) than 
such prolongation with respect to the General Linear group. Both algebras u^{S) and u(n, u^((^)) 
for all possible Jacobi symbols of rank 3 distributions were explicitly described [161 section 5] 
using the language of Algebraic Geometry , i.e. in terms of certain polynomials vanishing on 
certain tangential variety of a rational normal curve in a projective space and its secants. 
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The algebraic prolongation u^{S) of any symbol (5 of a curve of symplectic flags (with respect to 
conformal symplectic group) was described in [n\ subsection 8.3] using the representation theory 
of s[2. The construction of canonical frames for distributions of arbitrary rank with a given Jacobi 
symbol 6 and in particular the algebra u(n, u^((5)) will be studied in [21]. 

Example 4. Geometry of sub-Riemannian, sub-Finslerian, and other structures 
on manifolds via normal extremals. As in subsection II. 3| let U he a submanifold of TM 
transversal to the fibers and consider the time minimal problem associated with U . The extremals 
of this optimal problem are obtained from the Pontryagin Maximum Principle of Optimal Control 
Theory ([5U]). Assume that the maximized Hamiltonian of the Pontryagin Maximum Principle 

(3.25) H{p,q) = max p{v), q G M,p G T*M 

veU{q) 

is well defined and smooth in an open domain O C T*M and for some c > (and therefore for 
any c > by homogeneity of H on each fiber of T*M) the corresponding level set 

nc = {X£0: H{X) = c} 

is nonempty and consists of regular points of H (for more general setting see [3] or Remark 13.31 
below) . 

Consider the Hamiltonian vector field 1^ on Tic corresponding to the Hamiltonian H, i.e. the 
vector field satisfying i-^a = —dH, where a is the canonical symplectic structure on T*M. The 
integral curves of this Hamiltonian system are normal Pontryagin extremals of the time-optimal 
problem, associated with the geometric structure U, or, shortly, the normal extremals of U. For 
example, if Z// is a sub-Riemannian structure with underlying distribution D, then the maximized 
Hamiltonian satisfies H{p,q) = IIpI^,^^ Hg, i.e. H{p,q) is equal to the norm of the restriction of 
the functional p G T*M on Dq w.r.t. the Euclidean norm || ■ \ \q on Dg] O = T*M\D^. The 
projections of the trajectories of the corresponding Hamiltonian systems to the base manifold 
M are normal sub-Riemannian geodesies, li D = TM, then they are exactly the Riemannian 
geodesies of the corresponding Riemannian structure. 

Note that the case when U is a distribution without any additional structure does not sat- 
isfy these assumptions, because the corresponding maximized Hamiltonian is define on lA-^ only 
(and also identically equal to zero there). This is the reason why the distributions (without the 
additional structures) need a different treatment described in the previous examples. 

Further let Hciq) = Hc^T^M . Hdq) is a codimension 1 submanifold of T*M. For any X & He 
denote n(A) = Tx(Hc{t^{X))) , where vr : T*M M is the canonical projection. Actually n(A) is 
the vertical subspace of T\Hc, 

(3.26) n(A) = {eGrAHe:7r,(0=0}. 

Now define a sequence of subspaces n*(A), A G He, by the following recursive formulas for z < 0: 

(3.27) ni_i(A) := [t,Iii]{X), J_i(A) = J{X) 

Note that the symplectic form a induces the 2-form a on He and is exactly the line 
distribution of kernels of this form. Given a subspace A of A(A) denote by K'^ the skew-symmetric 
complement of W with respect to this form. By constructions n(A)^ = n(A). Using the operation 
of skew-symmetric complement we can define the subspaces nj(A) for i > as follows: 



(3.28) 



iii{X) = (n_,_2(A))^nn(A). 
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Further, for a generic point q € M there is a neighborhood U and an open and dense subset 
U of TT~^{U) n He such that for any z € Z dimnj(A) is the same for all X G U. Then the tuple of 
distributions (A,C, {Vijigz) on U with A = TT-Lc, C = M^, and Vj = Ilj satisfies properties (Pl)- 
(P4). As a matter of fact, in contrast to our previous examples, the vector field ^ is distinguished 
on C so we are interested not in the equivalence problem of the tuple (A,C, {Vjjjgz) but in the 
equivalence problem of the tuple (A, 1^, {Vijigz) We will say that the tuple is associated with 
the geometric structure hi by the symplectification. Since the vertical distribution 11 is one of the 
elements of this tuple, two geometric structures on M are equivalent if and only if the tuples 
associated with them by symplectification are equivalent. Besides, in the most of the situations 
the tuple (A, {Vijjgz) is recoverable from the corresponding flag structure (if one also take 
into account the distinguished parametrization on the curves of flags). 

The linearization ^'^ of the sequence of distributions {J7i}iez along the foliation Fol(i2^) of 
normal extremals at a leaf (a normal extremal) 7 is called the Jacobi curve of the normal extremal 

7. Since 7 is parameterized curve so that 7(t) = ^(7(0), the Jacobi curve Z'^ is parameterized 
as well. From ()3.27p . ()3.28p . and Remark 12.11 it follows that the curves of flags satisfy conditions 
(F1)-(F3) of subsection [231 

Remark 3.3. The same scheme works in more general situation, when the maximized Hamilton- 
ian is not deflned (for example, for sub-pseudo-Riemannian structures, deflned by a distribution D 
and pseudo-Euclidean norms on each space D{q)). Assume that for some open subset O C T*M 
there exists a smooth map u : O 1— ?• V such that for any X = {p,q) O the point u{X) is a critical 

point of a function hx : Vq M, where hx{v) p{v). Deflne H{X) = p(n(A)). The function H 
is called a critical Hamiltonian associated with the geometric structure U and one can make the 
same constructions as above with any critical Hamiltonian. 

Further, similarly to Example 3, given a symbol 5 of a parameterized curves of symplectic flags 
or, shortly, a parameterized symplectic symbol, one can deflne the notion of (5-regular curve. In 
the case when U is a sub-Riemannian or a sub-pseudo-Riemannian structure the set Hdq) = 
"He n r*M is a level set of a quadratic form and the corresponding Hamiltonian vector flelds 
depends algebraically on the flbers. In these cases, similarly to Example 3, for a generic point 
q (z M there is a neighborhood [/ of (/ in M and a parameterized symplectic symbol 5 such that 
any point from a generic subset of tt~^{U) n FWd is (5-regular and this symbol 6 is called the 
Jacobi symbol of the structure U at q. For more general geometric structures, it is possible that 
5i-regular and (52-regular points belong to the same flber Tidq) for two different parameterized 
symplectic symbols 61 and 62- In this case we can restrict ourselves to an open subset U{6) of Tic 
where all points are (5-regular for some parameterized symplectic symbol 5 and to proceed with 
the linearization procedure on this subset to get the canonical frame for the original structure 
from the canonical frame for the resulting flag structure (T9Jt, {d''}'ye<m) with parameterized flag 
symbol 5. Here 571 is the space of normal extremals in U. 

By analogy with Example 3 we say that a geometric structures U is said to be of maximal class 
if all curves xlZi do not lie in a proper subspace of A(7) for generic normal extremal 7. It was 
proved in pQ (although using a different but equivalent terminology) that any sub-Riemannian 
structure on a bracket-generating distribution is of maximal class. 

Further, let denote the degree —1 endomorphism of a 2m-dimensional graded symplectic 
space as deflned in Example 3, case a, after the formula (|3.24|) (i.e. the endomorphism generating 
the symbol of rank 2 distribution of maximal class on M"^"*"^). 
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From [43j it follows that for any sub-Riemannian structure on a bracket-generating distribution 
with Jacobi symbol 6 and, more generally, for any geometric structure U with the maximized 
Hamiltonian being well defined and smooth in the set U of 5-regular points and such that U is of 
maximal class the parameterized flag symbol 5 is a direct sum of endomorphisms of type Tm- 

More generally, fix two functions N^, N- : N — )■ N U {0} with finite support and assume that 
the parameterized symplectic symbol 6 is the direct sum of endomorphisms of type Tm and —Tm, 
where Tm appears N^(m) times and —Tm appears N^{m) times in this sum for each m G N. 
These symbols correspond to curves in a Lagrangian Grassmannian satisfying condition (G) in 
the terminology of the previous papers of the second author with C. Li ( |42l I43j ) and they may 
appear, for example, after symplectificatiom/linearization of sub-(pseudo)-Riemannian structures. 
Then from the results of [42 ^ [l3] or more general results of |17[ subsection 8.3.6] it follows that 
the non-negative part u^^'^^{6) ofu^'^^'^{6) is equal to so(A^4.(m), A^„(?ti)) and it is actually 

meN 

equal to the zero component u^'^^'^(5) of u^'^^^{5). 

Moreover, 5 is a nice symbol so that applying Theorem 12.11 to the corresponding flag structure 
we obtain a principal bundle P{d) over the space of normal extremals 9Jt in U{5) with the Lie 
algebra of the structure group isomorphic to 5o(A^+(m), A^_(m)) ©M(5. Moreover, this bundle 

meN 

P induces the principle bundle Pi{S) over U{6) with the structure group 0(A^+(m), A^_(?n)) 

meN 

(note that the bundles P{d) and Pi{6) coincide as sets). In particular, it gives the canonical 
(pseudo-) Riemannian metric on U{5), which immediate implies that the first algebraic prolon- 
gation of the pair (m,u^'^^'^{6)) is equal to 0, as in a (pseudo-) Riemannian case (here m is a 
commutative Lie algebra of the appropriate dimension). In other words, the canonical frame of 
Theorem \2.4\ (or Theorem \3 . 1\) applied to the flag structure (or the tuple of distributions) associ- 
ated with the geometric structure lA can he constructed already on the bundle P{5) . 

Note that, as already mentioned in [HldS], this type of constructions gives not only a canonical 
(pseudo-) Riemannian metric on U {5) but a canonical splitting of each tangent spaces to any point 
of U {6) such that each space of the splitting is endowed with the canonical (pseudo-) Euclidean 
structure. 

Finally, note that not any parameterized symplectic symbol is the direct sum of endomor- 
phisms of type Tmi and —Tm2 (or of type (D2) in the terminology of [171 section 7.2]), because 
there is another type of symplectically indecomposable degree —1 endomorphisms (type (Dl) 
in the same paper), which can be used in this direct sum. Similarly to the Jacobi symbols of 
rank 3 distributions these symplectically indecomposable endomorphisms are sums of 2 indecom- 
posable endomorphisms with respect to the General Linear group. The algebras u^'^^^{6) and 
u(m, u^'P*^''((5)) for arbitrary parameterized symplectic symbol will be described elsewhere. 



4. Proof of Theorem 12. 3t First prolongation of quasi-principle bundle 

Let P^ be a quasi-principle bundle of type (m, g^). Let IIq : — t- 9Jl be the canonical 
projection. The filtration {A*}j<o of induces a filtration {AQ}j<o of TP^ as follows: 

A[] = ker(no)*, 

^^■"^^ A^((/.) = {^;G^^pO:(^o)*^;GA^(^o(^))}, Vi < 0, G 
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We also set Aq = for all i > 0. Note that Aq((/3) is the tangent space at cp to the corresponding 
fiber of and therefore can be identified with the subspace of 0*^(111) via the map ^{(p) : 
T^{P%Uo{ip))) ^ LO, defined by (^3]). 

Besides, all spaces can be canonically identified with one vector space. For this take a 
subspace Mq of the space 0*^(1x1) C qI{q~^) such that the corresponding graded space grOJIo is 
complementary to grL^ in 0l(gr0~"'^), i.e. 

(4.2) 0[(gr0-i) =grL° egrA^o- 

Recall that by condition (2) of Definition 12.21 the space grL^ does not depend on 99, so the choice 
of TWo as above is indeed possible. Therefore 

(4.3) 0[(0-i) = lO ©A^o. 
for any 99 S P''. Let 

(4.4) := qKq-')/Mo, 

The splitting (|4.3p defines the identification Id^ between the space and the factor-space 
0[(0~^)/A^O) Id^ ■ ^ ■ The space has the natural filtration induced by the filtration on 
0^(0"^ ) • The identification Id^ preserves the filtrations on the spaces LO and LP. Note that by 
condition (3) of Definition 12.21 we have the following identifications: 

(4.5) 0°^grLO ^grLO. 

The space A^o is called the identifying space for the zero prolongation. 

Now fix a point ip £ and let ttq : A^^ip) / A}^'^ {^p) — )• Aq{p)) / A\^^ (ip) be the canonical projec- 
tion to the factor space. Note that Ho, induces an isomorphism between the space Ali{(p) / A^^^^ {(p) 
and the space A*(no((/7))/A*"'"^(no((/j)) for any i < 0. We denote this isomorphism by IIq. The 
fiber of the bundle over a point 7 G OJt is a subset of the set of all maps 



^e0Hom(0\A^(7)/A*+^(7)), 

i<0 

which are isomorphisms of the graded Lie algebras m = 0* and A*(7)/A*'''^(7). Let P^ be 

i<0 i<0 

the bundle over P^ with the fiber P^{ip) over (p> £ P^ consisting of all maps 
^ e 0Hom(0\ A^(v,)/A^+2(^)) 33 jjom(LO, A[J(v9)) 

i<0 

such that 



=n*o0 7r^o(^|g,, Vi<0, 



The bundle P^ is an affine bundle as shown below. Our goal in this section is to distinguish in a 
canonical way an affine subbundle of P^ of minimal possible dimension. 

For this fix again a point ip £ P^. For any i < choose a subspace fP C A^^ip) / A^^^"^ {ip) , 
which is a complement of ^^'^ {^p) / /A"^"^ {p) to Al(ip) / A}^"^ {ip): 

(4.7) A^(9.)/A^+2(^) ^ A^+i(9.)/Aj,+2(^) ® H\ 
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Then the map IIq o TTQ\ui defines an isomorphism between and A*(no(<^)) /A*'^^ (llo (</?)) . So, 
once a tuple of subspaces 7i = {i/*}j<o is chosen, one can define a map 

^ 0Hom(0\A^(v.)/A^+2(^)) 03 jjom(LO,AO(^)) 



i<0 



as follows 
(4.8) 



^'^\s' = (J^o°4\h') ^°^lg' ifi<0 

V'^Ilo =N<^)lLo))-'o(ld°)-^ 



Clearly ip = Lp^ satisfies ()4.6p . Tuples of subspaces T-L = {i?*}j<o satisfying (j4.7p play here the 
same role as horizontal subspaces (an Ehresmann connection) in the prolongation of the usual G- 
structures (see, for example, [32j or [5T] [section 2]. Can we choose a tuple {i/'}j<o in a canonical 
way? For this, by analogy with the prolongation of G-structure, we introduce a "partial soldering 
form" of the bundle and the structure function of a tuple T-L. The soldering form of is a 
tuple Oq = {'^o}i<0 5 where ojq is a g*-valued linear form on Aq(99) defined by 

(4.9) o.^(y) = v^-i(((no).(y))j, 

where ( (no)*(y)) is the equivalence class of (IIo )*(!") in A*(7)/A*+^(7). Observe that Ao+^((^) = 



ker Wq. Thus the form Wq induces the 0*-valued form ujq on Aq((^)/Aq^^(93). The structure function 
of the tuple % = {if*}j<o is the element of the space 



-2 



(4.10) A= Hom(g-i®5\0^)j Hom(g-i A 0-\ g'^) 

defined as follows. Let pr^ be the projection of Ao(9?)/Aq+^(v3) to Aq+^(</5)/Aq+^(93) parallel to 
W ( or corresponding to the splitting (|4.7|) ). Given vectors vi S g~^ and ^2 G g*, take two vector 
fields Yi and ^2 in a neighborhood of A in P'^ such that Yi is a section of Aq ^, ^2 is a section of 
Aq, and 

Y,{p) = p'^{vi), Y2{^)^^'^{v2)modAi+\^). 

Then set 

(4.12) CO (^1,^2) 4(p^t,{[Y,,Y2]{^)) 



In the above formula we take the equivalence class of the vector [yi,y2](v5) in ^o~^{v)/^o^^{f) 
and then apply pr^^. It is easy to show (see [H] section 3]) that Cy^{vi,V2) does not depend on 
the choice of vector fields Yi and 1^2 5 satisfying (|4.1ip . 
We now take another tuple 7i = {//*}j<o such that 

(4.13) Al{^)/Al+'{p) = Al+H.^)/Al+\p) e W 

and consider how the structure functions and are related. By construction, for any vector 
V € g* the vector ip'^{v) - ip^{v) belongs to Aq+^((/?)/Ao+^ ((/?). Let 

. . dcf /^o""' (^''(^) - v'^i^)) if ^ e 0^ with i < -1 



IdO o^^((^)((^W(^;)-(^«(^;)) if^;Gg- 
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Then e Hom(g*, g*^^) © Hom(g~^, L°). Conversely, it is clear that for any 



i<-l 



fe Hom(g^ 0^+1)0 Hom(g-\LO) 



i<-l 

there exists a tuple H = {//*}i<o, satisfying ()4.13p . such that / = /^^- In other words, the 
bundle is the affine bundle over such that each fiber is an affine space over the linear space 
Hom(0\g'+i) ©Hom(g"\LO). 

i<"l 

Further, let be as in ()4.10p . For any ip £ P^ define a map 

do : Hom(g^g^+l) ©Hom(g-\LO) ^ Ao 

i<-l 

by 

(4.14) dofivi,V2) = [f{vi),V2] + [VIJ{V2)] - f{[vi,V2]), 

where the brackets [ , ] are as in the Lie algebra m © g'^(m) (see ()2.10p ). The map do coincides 
with the Spencer (or antisymmetrization) operator in the case of G-structures (see, for example, 
|32j ) . Therefore it is called the generalized Spencer operator for the first prolongation ( at the point 
G P'^). Under the identification Id^ between spaces and we look at the operator do as 
acting 

(4.15) from Hom(g*, g*+i) © Hom(g-\ L°) to Aq. 

The following formula is a cornerstone of the prolongation procedure (for the proof see |4H 
Proposition 3.1]): 

(4.16) C7j = CO+5o/^^. 

Further the filtration p.3p on the spaces g^^ induces natural (nonincreasing by inclusion) 
■ 



filtrations {g ■ on each space g * with i > as follows 



(4.17) Qj ' = span{[?;i, [v2, [■ . . [vi-i,Vi], ...,]: Vk e g^^, -v < jk < -'i^,^jk > j} 

k=l 

—i 

For i < let grg* = g*'-' be the corresponding graded spaces, where g*'-' = gj/gj+i- Also, let 



]=tU 

-1 -1 



grm= grg^= g^ 



Then gr tn is a bi-graded vector space. Besides, the structure of a graded Lie algebra on m induces 
the structure of a bi-graded Lie algebra on grm with the Lie brackets [•, -Jgr defined as follows: If 
vi € g*^'-'^ V2 G g*^'-'^ , vi and V2 are representative of the equivalence classes vi and V2 in g*^^ and 
g*^, respectively, then 

(4.18) [vi,V2]gr-=[vi,V2] modgi;+|+i. 
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where [•, •] are the Lie brackets on m. Then for arbitrary vi and V2 from grm the Lie brackets 
[7;i,f2]gr are defined by bilinearity. 

Moreover, the Lie algebras m and grm are isomorphic: any hnear isomorphism / : gr0^^ 
can be extended to an isomorphism of Lie algebras m and gr m by setting: 

I{[V1, [V2, [,■■■ [Vi-l,Vi], ...,]) = [I{vi), [I{V2), [. . . [I{Vi-i),I{Vi)]gr,-- • ,]gr 

As / one can take J~^, where J : grg~^ is as in condition (2) of Definition 12.11 (with 

W = there). Any X G grg'^(m) C 0[(grg~^) can be extended to a derivation of the Lie 
algebra grm as follows: the operator J o X o belongs to and, in particular, can be extended 
to a derivation of the Lie algebra m. Let us denote this extension by Y. Then to define the desired 
extension of X we set Xv := oY o Jv for any t> G gr m. Besides, as in ()2.10p . one extends the 
structure of Lie algebra from grm to grm © grg'^(m). Moreover, its Lie subalgebra grm © gr 
is isomorphic to the Lie algebra m © g'^ and the isomorphism is given by 

(4.19) {v,X) ^ {Jv,J oX o J-^), t; G grm, X G grL°. 

Further, if A and B are vector spaces endowed with nonincreasing by inclusion filtrations 
{Aj}j^i and {Bj}j^z, respectively, then by analogy with ()2.ip define the filtration { (Hom(^, 2?))^}^^^ 
on Hom(A, B) by 

(4.20) (Hom(yl, B))^ = {X e Hom(A, B) : X{Aj) C Bj+k for any j G Z}. 

With this notation we can define the filtration on the domain space of the generalized Spencer 
operator do as follows: 

(4.21) I (Hom(g\g^+i)), © (Hom(g-\ )) J . 

J kez 

To define an appropriate filtration on the target space Aq of the operator Oq, first define the 
natural nonincreasing filtration of the spaces g~^ (8> g' and g~^ A g"^ as follows: 

-1 ^ — „„„„ r„, ^ . „. ^ „-i 

j 

j 

With this filtrations and the notation given by ()4.20p . we can define the following filtration on 
the target space of the operator do : 

-2 



g (8 QJ. = span{vi (g) W2 : fi G g^-^ , V2 G g*^, ji + j2 > j}, 
A gM = span{t;i A t;2 : f i G Qj^^,V2 G + j2 > j} 



(4.22) I (0(Hom(g-i©g\g^)),) ©(Hom(g-iAg-\g-i)) 



k 



i=—fi 



Note that directly from ()4.14p it follows that do preserves the filtrations ()4.2ip and ()4.22p of 
the domain and target spaces, i.e. it sends the kth space of filtration ()4.2ip to the feth space of 
filtration ()T22]) for any k eZ. 

Now as before assume that A and B are two filtered vector spaces endowed with non-increasing 
by inclusion filtrations {^fcjfcgz and {i?fc}fcGZi respectively. Let T : A B he a linear map 
preserving the filtration, i.e. such that T(j4fc) C Bfc for any k Z. Then to T one can associate 
the linear map gr T : gr A i-^ gr i? of the corresponding graded spaces such that gr T(a + A^^i) = 
T{a) + Bk+i. 

Let us consider the map gr do associated with the generalized Spencer operator do ■ Note that 
similarly to (j2.2p we have the following natural identifications for the domain space and the target 
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space of the map gr Oq (which are the graded spaces corresponding to filtrations ()4.2ip and ()4.22p 
of the domain and the target space of the operator Oq, respectively): 

(4.23) gr(0Hom(g\0^+i) ©Hom(0-i,LO) ^ Hom(gr 5^ gr 5^+^) Hom(gr 5-1, gr L"), 

i<-l i<~l 

(4.24) grA= 0Hom(grg~^ (8)grg*,grg*) ©Hom(grg~^ Agr0~\gr0~^) 

i<-l 

In particular, from condition 2) of Definition 12.21 it follows that the domain space of grc^o does 
not depend on a point £ P^. By the definition of Lie brackets [•, -Jgr and under identifications 
(lOSjl - dMI) we have 

(4.25) S[dof{vi,V2) = [f{vi),V2]gr + [Vl,f{v2)]gr - f {[vi, V2]gr) , 

Remark 4.1. Using the identification ()4.19p we can consider the operator gr9o as the operator 
from Hom(0*,g*^^) to ^0 satisfying the same formula as in ()4.14p . 

i<-l 

Now let us prove the following general lemma: 

Lemma 4.1. Let T : A B be a mapping of arbitrary filtered vector spaces A, B preserving the 
filtration. Let gr T : gr A — )■ gr S be the associated mapping of the corresponding graded vector 
spaces. Then the following three statements hold: 

(1) gr(ker T) C ker(gr T); 

(2) if C is any subspace in B such that 

(4.26) grCelmgrT = grS, 
then C + ImT = B; 

(3) under the assumptions of the previous items, the space grT~^(C) does not depend on C 
and coincides with ker(grT). 

Proof. 1) Suppose that a G A/^ and T(a) = 0. Then grT(a + Afc+i) = (o) + B^+i = and 
a + Afi-^i G gr ^ lies in the kernel of gr T. 

2) Let b be any element in B^^\ Then by assumption the element b + B^+i G gr B uniquely 
decomposes as {c + Ck+i) + {T{a + Ak+i)) for some elements c + Ck+i G gr C and a + Ak+i € gr A. 
Hence, we see that (6 — c — T(a)) lies in Bk+i- Proceeding by induction we get that b = c' + T(a') 
for some elements c' (z C and a' € A. 

3) Let a € T~^(C) H ^4^. Then gr T(a + A^+i) lies in gr C and, hence, is equal to 0. Thus, we 
have grT~^(C) C ker(grT). 

To prove the opposite inclusion ker(grT) C grT"^(C) we actually have to show that for any 
a (z Af^, satisfying T(a) € i?fc+i) there exists a' € Aj^. such that a — a' € ^fc+i and T(a') S C. 
For this let T^+i be the restriction of T to AC'"^). Then from (jOG]) it follows that ^rC^^"^) © 
ImgrTfc = B^^~^h Hence, by the previous item of the lemma we have 

Prom this and the assumption that T(a) € B^^~^^ it follows that there exist Cfc_i G C^^~^^ 
and fflfc-i € A^^~^^ such that T(a) = c^-i + T(afc_i). Therefore, as required a' one can take 
a' = a — Ok-i- Indeed, a' — a = a^^i € A^'^'^^ and T(a') = T(a — afc_i) = Ck-i € C. This 
completes the proof of the third item of the lemma. □ 
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Now fix a subspace 

A/'o C ^0 

such that 

(4.27) gr Ao = Im gr do gr A/'o. 

By analogy with G-structures and with principle bundles of type (m, g*^) the subspace Ao is called 
the normalization conditions for the first prolongation. From item (2) of Lemma 14.11 it follows 
that 

(4.28) Ao = lmdo+Mo. 
Given ip £ denote by P^{(p) the following space: 

(4.29) P^{ip) = {ip^ :% = {H']i<o satisfies (gTD and G A/'o}, 

where ip^ is defined by (|4.8p . Then from the formulas (j4.16p and (|4.28|) it follows that P^{'p) is 
not empty. Moreover, if (p"^ G P^{'^) for some tuple of spaces then G -P^(v') for another 
tuple of spaces % if and only if 

9o4,^gAAo. 

Here is acting as in gl5|). Therefore, P^{ip) is an affine space over the linear space 

(4.30) Ll ■= (9o)~'(AAo) C Hom(0\0^+i) eHom(0-i,LO). 

From item (3) of Lemma l4.1l it follows that the corresponding graded space gr-L^ (with respect 
to filtration (|4.2ip of the domain space of do) does not depend on the normalization condition 
Ao and coincides with ker gr . Taking into account Remark 14.11 we get that under identification 
(|4.19p kergr^o — S^j where is the first algebraic prolongation of the Lie algebra m g*^, as 
defined in (j2.12p . The bundle P^ over P^ with the fiber P^{ip) over a point G is called the 
first (geometric) prolongation of the bundle P^. 

Conclusion Given a subspace Ao C ^0 satisfying ()5.24p there exists a unique affine subbundle 
P^ of the bundle P^ with the fiber P^{^p) over the point ip that satisfies ()4.29p . A fiber P^{<p) 
is an affine space over the linear space C Hom(g*,g*+i) eHom(g-\L°). Moreover the 

j<-i 

corresponding graded space gic (with respect to filtration (|4.2ip ) is equal to the first algebraic 
prolongation g^ of the algebra m g'^ under the identification (|4.19p . 

Finally all spaces can be canonically identified with one vector space. For this take a 

subspace A4i of the space Hom(g*,g*^^) 0Hom(g~^,L°) such that the corresponding graded 

i<-l 

space grA4i is complementary to grL^ in Hom(grg*,gr g*^^) Hom(grg~^,gr L°), i.e. 

i<-l 

Hom(grg\grg'+^) 0Hom(grg-\grL°) = grLj, 0grA4i. 

i<-l 

By above, the space grL;^ is equal to g^, i.e. does not depend on (p, so the choice of A4i as above 
is indeed possible. Therefore 

Hom(g\g^+i)0Hom(g-\LO) = Li 0A^i. 

i<-l 
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for any ip £ P . This sphtting defines the identification Id^ between the factor-space 

(4.31) := ^ Hom(0S0^+i) ©Hom(0"\Li)^ /Ml 

and the spaces (which in turn are canonicahy identified with tangent space to the fiber 
P^((/9) of over if). The space has the natural filtration induced by the filtration on 
Hom(g*, g*"*"^) © Hom(g~^, L^). The aforementioned identification isomorphism preserves the 

i<-l 

filtrations on the spaces and L^^. The space Ali is called the identifying space for the first 
prolongation. 



5. Proof of theorem \2.'3\ Higher order prolongation of quasi-principle bundles. 

Now we are going to construct the higher order geometric prolongations of the bundle P^ 
by induction. Assume that all l-th. order prolongations are constructed for < I < k. We 
also set P~^ = DJl. We will not specify what the bundles P^ are exactly. As in the case of 
the first prolongation P^, their construction depends on the choice of the identifying spaces and 
normalization conditions on each step. But we will point out those properties of these bundles that 
we need in order to construct the {k + l)-st order prolongation P^~^^. First of all simultaneously 
with the bundles special filtered vector spaces P' are constructed recursively such that 

(Al) P° is as in 

-1 

(A2) P' is a factor-space of the space Hom(g*,g*+') © Hom(g*, P*+'); 

i<—l i=—l 

-1 

(A3) The filtration on P' is induced by the natural filtration on Hom(g*, g*+')©0 Hom(g*, P' 

i<—l i=—l 

which is given, similarly to (j4.2ip . by 

(5.1) |0(Hom(g^g^+O),© 0(Hom(g\P^+')) J , 

Ki<-1 i=-l ) k(,z 

where Hom(j4, P)^ is as in (j4.20p and the filtration on g* is given by (j4.17p : 
( A4) The corresponding graded space gr P' is naturally identified with the Ith. algebraic prolon- 
gation g' of the Lie algebra m + g'^. 

Before describing the properties of bundles P' note that the filtration on POJT induces naturally 
(by pullback) the filtration on each bundle P\ < I < k. Indeed, let 11/ : P' — )• p'~i be the 
canonical projection. The tangent bundle TP^ is endowed with the filtration {AJ} as follows: For 
I = —1 it coincides with the initial filtration {A*}j<o and for / > we get by induction 

A[ = kev{Ui), 

(5 2) 

Ali^i) = {veTxP' ■.m,veAl_^{Uii^i))y yi<i. 

We also set = for i > /. 

Below are the main properties of bundles P\ < I < k: 
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(Bl) The fiber of P\ < I < k, over a point ipi-i € will be a certain affine subspace of 
the space of all maps belonging to the space 

i-i 

Hom(0\ Ai„,(^,_i)/A;^l+^v^/_i)) ©0Hom(L^ Ati((/^i_i)). 

K-l 1=0 

Moreover, for each i, < i < I the restrictions fil^i are the same for all points tpi from 
the same fiber of P'; 

(B2) For < I < k ii ipi £ and tpi^i = Ili{ipi), then the points ^pi-i and ipi, considered as 
maps, are related as follows: if 

(5.3) 7^1 : Al{^i)/Ai+^+H^i) ^ AKv^O/Aj+'+^(^/) 
denotes the canonical projection to a factor space and 

(5.4) HI : AKv,,)/Ai+'+^(^0 ^ A]_,{M^O) / 
are the canonical maps induced by (11/)*, then 

for i < ipi^i Ig, = n'J„;L o nl^-^ o ipi j^, , 
for < i < / ^pl^l\L^ =iri^-i^oipi\^,. 



(5.5) 



Note that the maps IIJ are isomorphisms for i < and the maps vr^ are identities for i >0 
(recah that Aj = for i > /); 
(B3) For / > 1 the tangent spaces {=A\{(pi)) to the fiber P^ipi^i), where (fi-i = Ili{(pi), 
are canonically identified with certain subspaces L''^^__^ of the space Hom(3*,g*+') © 

i<-l 

-1 

Hom(g*,L^+'), which in turn are canonically identified with the space LK The obtained 
i=-l 

in this way canonical isomorphism between and A| ((/?/) will be denoted Id^^. Finally, 
Lpi\ii-i coincides with Id^^. 

Note also that for / > 1, the bundle is an affine bundle over p^~^ with fibers being affine 
space over the vector space LK In particular, the dimensions of the fibers are equal to dimg'. 

Now we are ready to construct the {k + l)-st order Tanaka geometric prolongation. Fix a point 
^k^^P^- Then 

k-l 

^fc € Hom(0\A*fc_i(99fc_i)/Att'i+n'^fc-i)) ©0Hom(L\A*,_i(v9fc_i)), 

i<-l i=0 

where ipk-i = Hkifk)- Let Tik = {-fr^}i<fc be the tuple of spaces such that Hj, = ^Pk{Q^) for i < 
and HI = ipk{L''-) for < i < A;. Take a tuple Hk+i = {^fc+i}j<fe of linear spaces such that 

(1) for i < the space Hl_^-^ is a complement of A'+^+'^{>fk)/^i^''^'^ifk) in (n[o4)-i(i?*) C 
Ali^k)/Al^''^\^k), 

(5.6) {Ul o nl)-\Hi) = Al+'+\^k)/Al+'+\^k) © H^,; 

(2) for < i < /c the space H^^^ is a complement of A^((^fe) in (n\)~^ {HI) , 

(5.7) ini)-\Hl) = Al{ipk)(BHl,,. 
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Here the maps vr^ and are defined as in ()5.3p and ()5.4p with I = k. 

Since A^'^'^"'"^(v9fc)/A5j^'^+^(v3fc) = kervr^ and is an isomorphism for i < 0, the map 11^ o 
TT^Iji^i^^ defines an isomorphism between Hl_^^ and for i < 0. Additionahy, by (j5.7|) the map 

(njt)*|j|^i defines an isomorphism between and ff^ for < i < k. So, once a tuple of 

subspaces Tik+i = {-ff|.+i}i<fci satisfying (|5.6p and (|5.7p . is chosen, one can define a map 

k 

^W.+i g Hom(0^AUy^.)/A^+'=+'(v9fc)) e0Hom(r,AUv'fc)) 



as fohows 











(5.8) 










ip'Hk+i 




= Id'' 

^Vk 



Can we choose a tuple or a subset of tuples Tik+i in a canonical way? To answer this question, 
by analogy with section U we introduce a "partial soldering form "of the bundle and the 
structure function of a tuple Tik+i- The soldering form of P'^ is a tuple Qk = {'^A:}j<fci where uj^ 
is a g*-valued linear form on A^((^fc) for i < and L*-valued linear form on A^,((^fc) for < i < k 
defined by 

(5.9) uUY) = ip^^(({UkUY) 



Here yiUk)*{Y)j is the equivalence class of (Hfc)*(y) in A|_-^(v3;,._i)/A*^"t'l^^(¥'fc-i)- By con- 
struction, it follows immediately that A^^^(99fc) = kertj|. So, the form induces the 3*-valued 
form 4 on Al(<^,)/Al+n¥'fc)- 



The structure function C!^^^^ of a tuple Tik+i is the element of the space 

A = I Hom(5-i ® 5', 0^+'=) I e f Hom(g-i ® q\ V+''^ 



(5.10) 



Hom(fl-i A Q-^,L^-^) e Hom(g-i ® L\L^-^) 



\i=0 / 

defined as follows: Let tT;'"' : Aj((^i)/Aj+'+2(93i) ^ A^((^/)/A*+'+^"''((^/) be the canonical projec- 
tion to a factor space, where —l<l<k,i<l. Here, as before, we assume that A; = for i > I. 
Note that the previously defined vtj coincides with tTj'"^. By construction, one has the following 
two relations 



(5.11) Au^.)/Ar+^(v..) = (e:=o<^'''(^^+i)j ® A^.'''^'(^^o/Ar+^(^.) if z < o, 

(5.12) Aliip,) = (e^-^ Hi^,) e Ali^,) iiO<z<k. 



Let prf be the projection of Al{ipk)/Al+''+^{ipk) to A{+''+\ipk)/Ai^''+^{^k) corresponding to 
the splitting ()5.1ip if i < or the projection of A^(99fc) to H^~l corresponding to the splitting 
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(|5.12p if < i < A;. Given vectors vi € and V2 € 0* take two vector fields Yi and Y2 in a 
neighborhood Uk oi tpk in such that for any (pk ^Uk, where 

fc-i 

(^fc G Hom(0^A*,„,(^,,((^fc))/Ai^+'i+l(^,,((^fc))) ©0Hom(L^Al,„l(^fc(^fc) 

i<-l i=0 

one has 



Uk,Yi{ipk) = ^k{vi), Uk,Y2{^k) = Mv2) mod A™i(nfc((^fc)), 

A: 



(5.13) 
Then set 

(5.14) C!^k+iivi,V2) = { 



iol^''(prYj:r{[Y,,Y2])) i^^<0 



prf_Yi([yi,y2]) ifO<i<A:. 



As in the case of the first prolongation, it is not hard to see that Cy^{vi,V2) does not depend on 
the choice of vector fields Yi and Y2, satisfying (|5.13|) . 
Now take another tuple Hk+i = {Hk+i}i<k such that 

(1) for i < the space Hi_^^ is a complement of A^+*^+^(v3fc)/A*^+'=+2((^fc) in (ff^o4)-i(i7^,) c 
Ai{^k)/A[^'-'\^k), 

(5.15) [iTk o nl)-\Hi) = Ai+''^H^k)/Al+'+\^k) H^i; 

(2) for < i < A: the space i^^+i is a complement of A^((/Pfc) in {Ul)~^{Hl), 

(5.16) (nl)-i(//D = A^(<^fc)e^^+i. 

How are the structure functions C^j and C~ related? By construction, for any vector f € fl* 

nk+i Hk+i 

the vector ^j^^+i (t;) - (^^fc+i (t;) belongs to A*^+''+^((^fc)/A*^+''+^((/?fc), for i < 0, and to A^((^fc), for 
<i < k. Let 

dcf f'^fc^''^^ (v) - (i;)) if f G 0* with i<-l 

fuk+iHu+S''^ - |(id^J-i (yi^+^{v) - if^^+^iv)) if G 0-1 OTV G LVith < i < A;. 

Then 

fc-i 

e © Hom(0^ 0^+^+1)© Hom(0\L^+'=+i)©0Hom(L\L*^). 

i<~k~l -fc-l<i<0 1=0 

In the opposite direction, it is clear that for any 

fc-i 

/e Hom(0\ 0^+^+1)© Hom(0\L^+'=+i)ffi0Hom(L\L'=), 

i<~k~l -k-l<i<0 1=0 

there exists a tuple Tik+i = {-f^fc_|_i}i<fc satisfying (|5.15p and (|5.16p and such that / = f-^^^^^fi^ ^■ 
Further, let Ak be as in ()5.10p and define a map 

k-l 

dk-. Hom(0',0^+'=+i)© Hom(0\L*+'=+i)©0Hom(r,L'=)^ A 

i<-k-l -k-l<i<0 1=0 
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by 

(5.17) 

'[f{vi),V2] + [vi,fiv2)] - f{[vi,V2]) if vi e Q~'^,V2 eg\i <-k- 1; 
dkf (.vi,V2) = < {fivi)){v2) - {f{v2)){vi) - f{[vi,V2]) if vi G Q-^,V2 eg\-k-l<i<0 
^-{f{v2)){vi) if e Q-^,V2 eL\0<i<k-l, 

Here in the first and in the second Hne the brackets [ , ] are as in the Lie algebra m © fl^Ctn) 
and in the second and the third hne in the expressions (/(ui))(f2) and {f{v2)){vi) we use the 
identification between the appropriate spaces L'-^^_-^ and from the property (B3) of the bundles 

Under this identification, we look on /(f i) as on an element of the space Hom(g*, Q^~^^) ® 

i<-k 

-1 

Hom(g^L*+'=), which gives the appropriate meaning to {f{vi)){v2)- Similarly, one gives the 

i=—k 

meaning to the expression {f{v2)){vi). 

Note that the map dk in general depends on the point ipk G P^. Also, for A; = this definition 
coincides with the definition of the generalized Spencer operator for the first prolongation given 
in the previous section. 

Similarly to the identity ()4.16p the following identity holds: 

(5.18) =Cl +dkU -u ■ 

A verification of this identity for pairs (^1,^2)1 where vi € g"^ and V2 G 0* with i < 0, is 
completely analogous to the proof of Proposition 3.1 in [5T]- For i > one has to use the 
inductive assumption that the restrictions are the same for all 93; from the same fiber (see 
property (B2) from the list of properties satisfied by in the beginning of this section) and the 
splitting (|5J2]1 . 

Recall that the domain and the target spaces of the map dk have natural filtrations induced 
by the filtrations on the spaces g*, i < 0, given by ()4.17p . Moreover, the map dk preserves these 
filtrations. 

What can we say about the associated map gr dk of the corresponding graded spaces? Using 
the identifications similar to (|4.23p and (|4.24p . identifications (|4.19p . and the property (A4) of the 
spaces above, the domain and the target spaces of the map gr dk can be identified with the 
spaces 

fe-i 

(5.19) Hom(0\ g^+'^+i) © Hom(g\ g'^) 
and 

(5.20) I Hom(g"^®g\g^+'=) 1 © Hom(g-i A g-\ g'^"^) © ( Hom(g-i g\ g'^"^) 1 , 

\j=-/i / \i=0 / 

respectively. Moreover, using these identifications and ()5.17p one gets that the map gr dk satisfies 



i<0 



(5.21) 

[f{vi),V2] + [vij{v2)] - f{[vi,V2]) if vi e Q~^,V2 € g*,i < 0; 
[vi,f{v2)] if vi G g"^^;2 G g', < i < A; - 1, 



dkf {VI,V2) 
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where the brackets [ , ] are as in the algebraic universal prolongation u(m, g*^) of the pair (m, g*^). 
Remark 5.1. Note that 

(5.22) / e ker grSfe ^ /Igi = 0, VO<i<A;-l. 

For the proof see section 3 of [H] (the map dk there coincides with the map gr dk here) . In other 
words, 

(5.23) ker gvdk C Hom(0\ g^+^+i). 

Moreover, directly from the definition, kergrc^o = Q^~^^, where g^~^^ is the {k + l)st algebraic 
prolongation of the Lie algebra m © g'', as defined in ()2.12p .D 

Now fix a subspace 

A4 c Ak 

such that 

(5.24) gr Ak = Im gr dk © gr TVfc . 

By analogy with G-structures and with principle bundles of type (m, g") the subspace A/fc is called 
the normalization conditions for the (k+l)st prolongation. From item (2) of Lemma l4.1l it follows 
that 

(5.25) Ak = lmdk+N'k- 
Given ipk € denote by P'''^^{ipk) the following space: 

(5.26) P''^\vk) = ■■ nk+i = {H'}i^k satisfies ^ and ([SlD and C^^^^ G A4}, 

where 99^*=+! is defined by ^Mj. Then from the formulas (f538l) and (fOSj) it follows that P''+'^{ipk) 
is not empty. Moreover, if (^^'=+1 G P^~^^{ipk) for some tuple of spaces Hk+i, then 99^'=+! G 
P^~^^{ipk) for another tuple of spaces Hk+i if and only if 

Therefore, P''+^{ipk) is an affine space over the linear space 
(5.27) 

L^f :=(9fc)-'(A4)C Hom(g^g^+*^+l)© Hom(g\ r+'^+i) © Hom(L\ L*^). 

i<-k-l -k-l<i<0 1=0 

From item (3) of Lemma l4.1l it follows that the corresponding graded space gr L^^^ (with respect 
to the filtration of the domain space of dk) does not depend on the normalization condition Mk 
and coincides with kergrdk, which according to Remark 15. II can be identified with g'^"'"^. Besides, 
from (f^THh it follows that if / G L^j^, then /j^, = for all < z < /c. This imphes that 

^'t'c Hom(g\g^+'=+i)© Hom(g\ L^+'^+i). 

i<-k~l -fc-l<i<0 

Moreover, this implies that for any i, < i < k, the restrictions (fk+il^i are the same for all points 
(fk+i from the same fiber P''~^^{(pk). Note also that by ()5.8p the restriction (fk+ili'' coincides with 
the identification Id^^. The bundle over P'' with the fiber P^^^tpk) over a point (pk G P 

is called the (A; + l)st (geometric) prolongation of the bundle P". 
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Further, all spaces L^"^^ can be canonically identified with one vector space. For this take a 
subspace of the space ^ Hom(0*, 0*+''+^) © ^ Hom(0\ such that the 

i<-k-l -k-l<i<0 

corresponding graded space gr A^i is complementary to grL^ in 

Hom(gr0\gr0^+^-+i)© Hom(gr0\grL^+'=+i). 

i<~k-l -fc-l<j<0 

Hom(gr0*,gr0*+^) ©Hom(gr0"\grL°) = gr © gr A^^+i. 

i<-l 

By above, the space grL^"*"^ is equal to q'''^^, i.e. does not depend on 99^, so the choice of A^^+i 
as above is indeed possible. Therefore 

Hom(0\0*+'=+i)ffi Hom(0\r+'=+i) = L^f ©A1fc+i. 

i<~k~l ~k-l<i<0 

for any ipk G P^. This splitting defines the identification Id^^"^ between the factor-space 

(5.28) L'=+1:=( Hom(0\0^+'=+i)© Romig\ 1'+"+')] / Mi. 

Yt<-fe-i -fe-i<t<o / 

and the space L^+^ The space L^~^^ has the natural filtration induced by the filtration on 
Hom(0*, 0*"'"'^"'"^) ffi Hom(0*, L*"'"'^"'"^). This identification preserves the filtrations 

i<~k-l -fc-l<i<0 

on the spaces L^~^^ and L^'l^- The space A4k+i is called the identifying space for the {k + l)-st 
prolongation. 

By our constructions the space L'^'*'^ is canonically identified with tangent space to the fiber 
P^^^{i.pk) at any point '.pk+i. Denote the identifying isomorphism by Id^"^^^. In this way we finish 
the induction step by constructing the space L^^^ and the bundle P^~^^ satisfying properties 
(A1)-(A4) and (B1)-(B3) above for / = A; + 1. 

Finally, assume that there exists T > such that 0' 7^ but 0^''"^ = 0. Since the symbol m is 
fundamental, it follows that 0' = for aW I >l . Hence, for all / > T the fiber of P' over a point 
Ai_i € p'-i is a single point belonging to 

Hom(0\Ati(</^i-i)/Ajll+Hv'^-i)) ® 0Hom(r,Ati(^i-i)). 

i<~-l 't=0 

where, as before, /i is the degree of nonholonomy of the distribution A. Moreover, by our as- 
sumption, A; = if / > / and i >l. Therefore, \i I = I + fi, then i -|- / -|- 1 > Z for i > — /i and the 
fiber of over P' is an element of 

Hom| 0^©0AT,,_,P'-M . 

In other words, P'+z^ defines a canonical frame on p'+z^^i. But all bundles P' with / > I are 
identified one with each other by the canonical projections (which are diffeomorphisms in that 
case) . This completes the proof of Theorem 12. 3[ 
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